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Abstract 

The purpose of this study was to describe the depth and accuracy of sixth grade students! content understandings as a result of engaging in information-seeking 
activities using Web-based resources in the University of Michiganis Digital Library. Eight student pairs received scaffolded pedagogical support from classroom 
teachers while completing four on-line inquiry units (astronomy, ecology, geology, weather) during a nine month period. Emerging understandings were assessed 
through a content analysis of learner final products and semi-stmctured interviews conducted during the school year. The analysis provided insight into both the depth 
(recalling information, offering explanations, articulating relationships, extending explanations) and accuracy of learner understandings (inaccurate, none, partial, 
accurate). Results of this study provide an empirical perspective on the use of World Wide Web resources in science classrooms. 



Assessing the Nature of Learneris Content Understandings as a Result of Utilizing On-line Resources 

Educators in the 20th century have seen a number of technological innovations and the application of "newfangled tools" into science classrooms across the United 
States. Examples of these educational tools include filmstrip projectors, televisions, hand-held calculators, videotape cameras and players, and more recently the use of 
stand-alone, networked, and portable microcomputers. Advancements in the performance of microcomputers and associated software have allowed desi^ers to 
construct simulations, lab interface apparatus, communication and collaboration tools, tutorials, and a variety of student assessment and evaluation materials. Although a 
number of these innovations have seen widespread implementation in classrooms, their appropriate application and impact on student leanung continues to be a source 
of constant debate. The most recent innovation for classrooms, the use of the World Wide Web, offers yet another opportunity for enhancing the ways in which 
teachers teach and learners learn ~ although this claim has been left largely unexamined. 

The increased reliance on the World Wide Web for providing educational experiences to K-12 learners requires the immediate attention of the research community. 
Policy makers and educators are moving forward at a rapid pace to incorporate this resource in classroo^ with little concrete guidance from theory on learning or 
empirical studies. Although a number of scholars have presented various arguments regarding its potential for learning, the literature falls short of providing substantial 
evidence to support their claims. For example, advocates (Bame & Presti, 1996; Kinzie, Larsen, Burch, & Boker, 1996; Ryder & Graves, 1997) speak to the potential 
of the World Wide Web as an instructional tool for classrooms. General statements include: "The Internet and WWW provide us with the potential to change the 
nature of learning. We can use it to increase access to effective instructional materials in a variety of media." (Kinzie et al., 1996, p. 59); "Access to the Internet 
provides an opportunity to enhance students! reading and writing proficiency and to promote their skills in information gathering and problem solving. (Ryder & 
Graves, 1997, p. 244); "By its nature, the World Wide Web is a tool ideally and uniquely suited for the advancement of education." (Barrie & Presti, 1996, p. 371). 
Others imply more definitive possibilities: "Network collaborations, mentoring programs, access to supercomputers and instruments and, of course, access to the 
decentralized resources on the Internet, have enriched education." (Tinker & Haavind, 1997, p. 1 ) 

cO However, while a large collection of literature speaks to the positive nature of the use of World Wide Web resources in science classrooms, other sources (Lookatch, 
1995; Maddux, 1996; Winebrener, 1997; Stoll, 1998) remain pessimistic on its application as an instructional tool. Winebrener (1997) suggests "surfing the World 
Wide Web" is a "current educational panacea, bandwagon, magic bullet, fad..." and recommends educators "proceed with caution" before integrating it into the 
classroom. He describes how students can become mislead by viewing falsified information posted on the Web, frustrated due to an inability to locate specific 
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information easily, and confused with varying forms of navigation found on individual pages. Winebrener (1997) describes his frustration with the lack of consistency 
in design and presentation of Web-based resources: "The Internet flies in the face of this philosophy [common principles of design]. Every site is unique with 
different buttons, dissimilar organization, unique paths to follow, and disparate information presented." (p. 21) Maddux (1996) supports these cautions and describes 
other problems related to the open-ended nature of the World Wide Web: 

... it is intellectually appealing to think that hypermedia and open-ended exploration of data are more consistent with the way children think and learn 
than is linear exploration on which traditional print media is based. However, this assumption has yet to be verified through research, and I suspect 
that the truth is very much more complex than this. (p. 28) 

Still others, maintain that research has been unable to prove using a computer or any other technology improves learning (Lookatch, 1995). In his critical ^sessment of 
educational technology studies, Lookatch makes a number of comments: "...[there is] a blind belief in a new and novel machine wltich students find more interesting 
and easier to accept than a talking head in front of the class." (p. 6); "There are no unique educational benefits from multimedia or its attributes." (p. 6); "multimedia is 
not the great equalizer. Access to technology is clearly inequitable and will continue to be for the foreseeable future." (p. 7). His harsh critique of educational 
technology is founded on a belief that a majority of current technology research contains inappropriate methodologies for controlling "a host of conditions that may 
account for the observed impact on learning." Lookatch (1995) suggests a teacheris pedagogical approach contributes to gains in student learning and achievement 
while the sole implementation of technology does not. 

The complexity of these issues and messages continue to provide uncertain guidance to policy makers and educators who will continue to implement these tools in 
classrooms across the United States. Clearly, there is a need to broaden the scope of research with students! use of the World Wide Web and provide insight into its 
impact on teaching and learning. This research explored this learning environment through a year-long study of middle school learners as they engaged in 
information-seeking activities using on-line resources. Of importance were the content understandings that emerged as a result. 

Study Rationale 

Given the variety of literature for both positive aspects and challenges associated with using the World Wide Web in K-1 2 classrooms, it is critical to provide empirical 
evidence of its impact in schools. A number of instructional and learning questions beg empirical support: How does intpaction with on-line resources impact student 
learning and understanding? How do learners make sense of information found on the Web? How do pedagogical practices influence the construction of 
understanding? 

Early studies (Hoffman, 1997a; Hoffman, Kupperman, & Wallace, 1997; Lyons, Hoffman, Krajeik, & Soloway, 1997; Wallace, 1997; Wallace & Kupperman, 1997) 
conducted as part of the University of Michigan Digital Library and Middle Years Digital Library projects concluded students have dfficulty locating and using 
resources in on-line environments. Specific challenges cited in their empirical studies were: Students have difficulty locating and taking advantage of information 
(Lyons et al., 1997; Wallace, 1997; Wallace and Kupperman, 1997), students often become lost and disoriented while navigating (Hoffman et al, 1997), students 
require a substantial amount of support to frame their activities (Hoffman et al., 1997; Lyons et al., 1997), students do not evaluate the resources they find (Wallace, 
1997; Wallace & Kupperman, 1997), and students have difficulty using Web tools beyond basic operations (Wallace, 1997; Wallace and Kupperman, 1997). 

However, these studies were conducted over short periods of time (1 to 2 weeks) and utilized limited sources of data. This study utilized an extended perspective (4 
units during the school year), a variety of data sources, and considered pedagogical influences to fully understand the dynamics of classroom teaching and learmng 
with the World Wide Web. Using fine-grained data collection and analysis techniques, this research provides substantial insight into how learners construct 
understandings from on-line resources. 



Research Questions 

Based on this rationale, this year-long study focused on the effect of utilizing the Internet and World Wide Web as a technolo^cal tool to enhance student learning 
and content understanding. Using on-line resources as a means to support inquiry, it provided insight on the nature of learoersi science content understandings -- 
understandings that resulted from the processing of science facts, concepts, and processes demonstrated by a leamen's ability to explain their new conceptualizations, 
articulate it in problems or situations, and extend it to new or novel contexts. This study focuses on the following question: What is the nature of learners! science 
content understandings as a result of utilizing on-line resources? To provide a more in-depth perspective on this study, two subquestions were drawn from this broad 
question: 



1 . What depth of content understanding (recalling information, offering explanations, articulating relationships, extending explanations) do 
learners possess and communicate as a result of utilizing on-line resources? To what extent are these made visible in learners! products and 
conversations? 

2. How accurate are the content understandings (inaccurate, none, partial, accurate) that learners possess and communicate as a result of 
utilizing on-line resources? To what extent are these made visible in learners! products and conversations? 

This research assists in lending evidence to questions regarding the value of the World Wide Web as a viable media for learning by providing an empirical perspective 
on students! emerging content understandings. Also, it builds upon previous studies and considers the impact of pedagogical factors in the analysis -- early studies 
failed to fully explore the influence these have on student learning. In addition, this study follows learners throughout the entire school year ~ previous studies did not 
measure changes that could have occurred through the year. Most importantly, this study begins to provide support to theoretical claims made regarding the World 
Wide Web and the impact it has on teaching and learning. 



Theoretical Underpinnings 

The foundation of this study draws upon interwoven contexts framing the environment in which learners participated. Students engaged in on-line information-seeking 
activities, partnered with technological tools designed to support inquiry, and received substantial scaffolding. As a result, students were expected to develop new 
science content understandings. These can be framed with two contexts: conceptual and technological. 



Conceptual Context: 

A number of researchers describe information-seeking in electronic environments similar to the World Wide Web. These descriptions suggest information-seeking is 
a "special case of problem solving" (Marchionini, 1 989) where learners recognize and interpret an information problem, establish a plan of search, conduct the search, 
evaluate the results, and "use" (Kuhlthau, 1993; Wallace, 1997) information to solve a problem. McNally & Kuhlthau (1994) describe how information-seeking 
consists of both undirected and highly directed activities - undirected searching that leads to unexpected links, or discrepant events related to their topic, and highly 
directed searching for the purpose of finding specific information. They suggest students move through predictable stages (initiation, selection, exploration, 
formulation, collection, presentation) as they engage in these activities and claim students progress from "ambiguity to clarity, from seeking general infoimation to 
seeking specific information." Kuhlthau (1993) describes how learners "construct their own points of view or understanding of a topic or problem" and increase their 
interest and confidence as they progress from initial conceptualizations of questions to the conclusion of the process. 



In addition to information-seeking, a variety of literature (Nickerson, 1995; Perkins, Crismond, Simmons, & Unger, 1995; Perkins & Simmons, 1988; Talsma, 1997) 
defines the nature of science content understandings. Perkins & Simmons (1988) define content understanding as not only a recollection of facte and definitions 
associated with a particular subject area, but the utilization of mapping schemes to associate concepts with referents and strategies for memorization and recall (i.e. 
metacognitive strategies). A perspective of content understanding is further refined by Nickerson (1995) as a "matter of degree" that an individual understands 
something (i.e. concept, principle, structure, process) at a relatively deep level. These evidences include ^e ability of a learner to: explain to the satisfaction of an 
acknowledged expert, apply knowledge appropriately in various contexts, produce appropriate qualitative representations, make appropriate analogies, repair 
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malfunctions, and predict the effects of change in structure or process. Talsma (1997) suggests that evidence of content understanding requires "more than producing 
verbal answere on cue" but instead involves transforming knowledge into "thoughtful understandings." She states: "...suppose that a learner can explain a concept in 
their own words, can exemplify its use in fresh contexts, can make analogies to novel situations, can generalize the law, recognizing other laws or principles with the 
same form, etc. When learners go beyond the information given, then we recognize that they can understand." (p. 2) 

Perkins, et al. (1995) provides theoretical support for the complicated nature of knowledge construction by describing "outeide" and "inside" underst^dings and ^e 
necessary cognitive resources learners must possess to develop these understandings. Outside understandings are made evident to others in terms of "overt behaviors" 
as learners communicate or act in three ways: 

1. Offering Explanations. Displaying understandings by giving examples, highlighting critical features, and responding to new situations. Learners who 
simply present facts and describe phenomena without the ability to explain concepts, clearly lack understanding. 

2. Articulating Richly Relational Knowledge. Expressing understandings with explanations constructed of knowledge linked from related aspects of a concept 
or phenomena and illustrating it in a single context. Learners who utilize sparse knowledge involving one simple rule would display poor understanding. 

3. Displaying a Revisable and Extensible Web of Explanation. Demonstrating understanding by revising and extending explanations beyond the original 
source of information (i.e. textbook, lecture, video, experiment) to new contexts or situations. Learners who simply "parrot" explanations and cannot extend 
them to new contexts lack robust understanding. 

Perkins et al. (1995) describe how these "outside" understandings, especially those tied to new contexts are frequently "fragile" and can grow or diminish in response 
to subsequent learning, discussion, or evidence. 

The construction of "inside" understandings involve learners exploring a rich conceptual network described by Perkins et al. (1995) as an explanation structure: "A 
rich network of explanatory relationships that are encoded mentally in any of the many ways the mind has available -- through words, images, cases in point, anecdotes, 
formal principles, and so on." (p. 74) These structures, or "substrates" provide more than a memorized explanation as learners are able to extend and revise their 
understandings. Although some components of explanations may be "well rehearsed," other parts of the structi^ are "novel" and "created on the spot" pemutting 
learners to extend their substrate. In addition to possessing a stable substrate on which to build new understandings, students require ongoing conceptual activities (i.e. 
conversation, exploration, instruction) to sustain active learning. In summa^, Perkins et al. (1995) suggests explanation structures are not static, but expand as learners 
engage in topics and contract as they set topics aside, leaving new associations on the substrate to increase the likelihood of future reconstructions and extensions. 

This study uses, in an exclusive sense, the perspective presented by Perkins et al. (1995). The notion of making depth of understanding public through explanation, 
articulation, and extension, provides a concrete method of classifying learners "outside" understandings. However, it is understood that these newly constructed 
understandings are fragile and are subject to modification based on new information and additional learning experiences. 



Technological Context 

A number of prominent scholars define the realm of technological tools for use in educative environments. Th^e include early conceptualizations by Tinker ( 1 978) 
describing how computers could be used for assisting students with data analysis, simulating laboratory expert rnents, and as a measurement processing instrument. 
More current literature suggests that interactive vide(xlisks, telecommunications, microprocessor-based laboratories, and software applications for modeling, 
visualization, and simulating can "support learners as they solve complex and ambiguous problems by providing access to data and information, and opportunities to 
collaborate, investigate, and create representations." (Marx, Blumenfeld, Krajcik, & Soloway, 1997). Salomon, Perkins, & Glqberson. (1991) claim these technologies 
have the potential to aid in cognitive or mental processing, can support intellectual performance, and enrich the minds of individu^s. Pea (1985) refers to these 
computer tools as "cognitive tools" or "technologies of the mind." He suggests computer tools allow learners to accomplish certain tasks faster and with less effort, 
they can also redefine and restructure learning tasks. Bereiter & Scardamalia (1987) propose intellectual partnerships with technological tools can leave a cognitive 
residue in the form of a "generalized ability" for self-regulation and guidance. 

In addition to these technological tools, a large body of literature describes the structure of hypermedia-based learning environments like the World Wide Web and 
theoretical advantages it has for learners. Conclin (1987) defines hypermedia as a method of building systems for organizing, structuring, and accessing information 
around a network of multimedia nodes connected together by links. These hypermedia systems permit the integration of text, graphics, audio, animation, and video into 
a "multidimensional learning environment" (Shepardson & Britsch, 1996). Salomon et al. (1991) suggest learners can develop "intellectual partnerships" with 
hypermedia-based programs as it "assumes part of the burden of information processing" as information is located, retrieved, and presented in a common format on 
the computer screen. Marchionini (1988) suggests hypermedia systems allow users to move "easily among vast quantities of information according to plan or 
serendipity" allowing freedom from the "linear, highly directed flow of printed text." Heller (1990) describes how hypermedia learning environments provide 
opportunities for the "exploration of alternatives" that can result in the "understanding of relationships that were previously unrecognized." 

Browser software takes advantage of hypermedia design principles to make information easily accessible on the World Wide Web. Users simply click on text 
(hypertext), graphical buttons (icons), or images (image maps) to navigate through various networks, sites, and pages within sites in a non-linear manner. Through the 
utilization of these design principles, current browser software has enabled a wide range of users to access diverse sets of resources on the World Wide Web. The 
intuitive nature of this click and browse interface has certainly contributed to the popularity of the World Wide Web as a source for information retrieval. 



A large body of literature (Gordin, Gomez, Pea, Fishman, 1 996; Lea & Scardamalia, 1 997; Linn, 1 996) describes the theoretical advantages of using the World Wide 
Web for enhancing student learning. Linn (1996) suggests students can utilize the Internet as "a resource for developing a cohesive, linked, connected, and integrated 
understanding of science" (p. 34) when appropriately supported in knowledge building environments. Gordin et al. (1996) describes how the networked structure of 
the World Wide Web can "play a vital role in the formation and successful operation of learning communities." (p. 2) In addition. Lea & Scardamalia ( 1 997) suggest 
specific Web-based collaborative and database tools can "support knowledge construction and progressive inquiry in a dynamic way." (p. 1) 

In addition to the theoretical affordances hypermedia-based hold for learners, access to primary resources via networks provides unique opportunities for the 
construction of new understandings. A number of researchers (Barrie & Presti, 1996; Gordin, Gomez, Pea, & Fishman, 1996; Ryder & Graves, 1997; Tinker, 1996) 
have explored potential uses for the World Wide Web in science classrooms. Descriptions of these applications range from general accounts of broad activities to 
specific classifications of groups of resources. Barrie & Presti (1996) present three ways educators can utilize Web-based resources for instructional purposes: 
Providing both general and specific informational resources to students, incorporating the Web as an integrated interface for distance learning or "virtual classroom," 
and supplementing conventional content in the classroom. Ryder & Graves ( 1 997) describe other applications: Utiliang the Web to access a broad range of 
information types in both textual and graphical formats, communicating with others via email, newsgroups, and live video, and obtaining instruction^ computer files 
and software. Tinker (1996) provides further insight and describes three classes of applications for supporting core mathematics and science educational goals: 
Accessing a diverse set of data and information that "no school could afford to acquire for its library," participating in "network-based communities* to promote 
student collaboration on projects, and obtaining on-line professional development materials for teachers. Other perspectives from Gordin et al. 0996) include three 
categories of recent applications of Web-based technology in K-12 learning situations: Accessing digital resources in the form of on-line libraries, museums, 
government information, curriculum and activities, and indices, collecting "raw data*' and accessing a variety of analysis tools to assist with the interpretation of 
information gathered on-line, and collaborating between students and work-based learning communities. 



Still others attempt to organize applications of the World Wide Web into more specific categories. Hoffman (1997b) suggests the Web has particular application in 
secondary science classrooms for: Collaboration and data sharing, distance learning, mentoring and tutoring, remote instrument control and data collection, virtual 
laboratory experiences, virtual field trip experiences, authoring and publishing, and research with primary sources. He suggests perhaps the greatest strength of the 
World Wide Web is the diverse range of primary resources available on-line from government (.gov) and educational (.edu) communities. These resources differ from 
other sources on the Web as they originate from major institutions, are often based on original or historical research and analysis, and can be interpreted as factual in 
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nature. Students can take advantage of primary resources in a variety of ways. These include information gathering for research papers, reports, topic summaries, 
presentations, and other content-rich activities. 

Wallace, Krajcik, & Soloway (1996) describe the benefits of Web-based resources (e.g. primary on-line resources) in terms of "characteristics'* that are not 
necessarily unique to the World Wide Web, but are "quite different from the resources normally available in K-12 schools." These characteristics are: 

1. Content is current. Using on-line resources on the World Wide Web, students can obtain up-to-date information in many content areas, 

2. Content can be from primary resources. Students can often use the same data and information sources as scientists do. 

3. Content is comprehensive. In typical libraries used by secondary school students, only a small subset of popular and scholarly material on a given subject is 
available. The Web expands the range of content enormously, and in the future digital libraries may do so even more, 

4. Resources are represented in various formats. In particular, information is available in digital form for easy manipulation and use by students. Video and 
sound provide new information (for example, dynamic views of the ozone holes and resonant sounds of orchestra halls), and new ways of receiving 
information. 

5. Students can publish on-line. Students can produce content to share with others on-line. 

6. Students can collaborate on-line. They may have access to other students doing similar activities or to experts in their field of study. 

7. Content is readily accessible. Information is easily obtainable from a single point of access. That is, students can acquire a variety of information through a 
single computer. 

A variety of literature (Flake, 1996; Kay, 1996; Tinker, 1996a, 1997; Wallace et al., 1997) supports the use of primary sources on the World Wide Web for 
research-based activities. Wallace et al, (1997) in their study of Web-based digital libraries discuss the importance of students asking "personally meaningful 
questions" and engaging in information-seeking activities using digital resources. Tinker (1996a) describes how new technologies, like the Web, can support student * 
inquiry because it "extends the capacity of students to undertake investigations" while accessing information resources. Flake (1996) discusses how students should 
be engaged in sense-making, problem solving, and open investigations using on-line resources, while developing skills for accessing knowledge. 

A large segment of this study involves students utilizing primary resources for their information-seeking activities. The availability or non-availability of these 
resources have the potential to impact this study to a substantial degree. Primary resources having particular relevance to student questions may impact their capacity to 
engage in investigations. However, the likelihood of having primary resources available for all student topics and questions are slim — how will learners respond when 
their question-asking is not supported by on-line resources? 

In summary, this section provided a review of literature in respect to the contexts (conceptual, technological) that frame this study. Each of these contexts were critical 
to explore using current literature in an effort to establish a framework in this year-long study. 

Learning Environment 

The purpose of this study was to explore the nature of leamersi emerging content understandings within a framework of on-line inquiry through information-seeking. 
Students engaged in week-long investigations with the goal of developing a solution or answer to a question they have posed. The participants of this study were 
immersed in a learning environment designed to support question asking, resource gathering, and the construction of new understandings. This environment, designed 
by the University of Michigan includes a number of interrelated components: The University of Michiganfs Digital Library, Artemis - The Interface to the Distal 
Library. Middle Years Digital Library - On-line Learning Materials, and Tactics aad_Strategies For Leading On-Line Investigations . Although explained individually 
below, they comprise a coherent whole for attempting to engage students (and teachers) in on-line inquiry. 

The University of Michiganfs Digital Library 

The University of Michigan Digital Library (UMDL) Project is an ongoing effort of the Digital Libraries Initiative. The projectfs goal is to create an architecture and 
software infrastmcture for development of a digital library open to multiple heterogeneous collections. Wallace et al. (1998) describe how the structure and contents of 
the collections are "constrained only by their ability to communicate with an agent in the UMDL infrastmcture about the contents, search protocols, and other technical 
information." (p. 1) Within the larger agent architecture, multiple smaller agents find information, negotiate prices for copyrighted information, browse collections, and 
register materi^s. Although the UMDL project currently includes test-bed earth and space science collections, other collections can be envisioned for specific user 
groups. 

The UMDL collections for this study include a journal collection from UMI (Proquest Direct); a Web Collection (registered World Wide Web sites for 6th and 9th 
grade students), and a set of reference materials (Encyclopedia of Science and Technology, Grolierfs Encyclopedia Americana, Websteris Dictionary, WordNet 
Dictionaiy/Thesaums). The Web Collection consists of over 750 World Wide Web sites and pages identified by librarians as primary resources for middle and high 
school students in earth and space science classes. University librarians populated the Digital Library by locating, evaluating, and abstracting resources with both 
readable and appropriate content. In addition, librarians located resources that would take advantage of a Web-based environment - those containing engaging content, 
high levels of interactivity, animated illustrations, and on-line forms for communicating to Web authors. These sites are registered in the UMDL using a controlled 
vocabulary permitting students to initiate searches for accessing the collections. 

Students used the University of Michiganfs Digital Library during the course of this study to access digital resources related to their questions. This environment 
hoped to provide a concise set of collections that stand in contrast to those utilized in previous studies (Lyons, Hoffman, Krajcik, & Soloway, 1997; Wallace, 1997; 
Wallace & Kupperman, 1997). These early studies permitted students to access resources from the entire World Wide Web - and they were often unsuccessful in 
attempts to locate appropriate materials for their investigations. It was expected the UMDL had a potential to influence the degree to which students engage in 
information-seeking activities and the content understandings they develop as a result of using organized, age-appropriate collections. 

Artemis - The Interface to the Digital Library 

A major component of this study was student interaction with Artemis (Figure 1), a Java-based interface to the UMDL accessible from the World Wide Web. This 
interface was designed to provide specific cognitive support for learners as they engage in information-seeking activities (Wallace et al., 1998). Artemis allows students 
to focus on the task of inquiry rather than concerning themselves with organizing their workspace, recalling pass searches and sites, and recording addresses of useful 
sites. 



Fi gure 1. Artemis Interface Main Screen 
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Driving Question folders (Figure 2) provide support by encouraging students to reflect on useful sites they find (as well as their question) before placing the resources 
in folders. 

Fi gure 2. Artemis Driving Question Folder 




Broad Topics (Figure 3) provide support by helping students generate keywords, recall prior knowledge, and view structures of a particular content areas before 
initiating queries to the Digital Library. 



Fi gure 3. Artemis Broad Topics List 




Collections (Figure 4) also support students by providing useful resources pertinent to their Mving Question. This allows students to focus on the contents of the 
resource, evaluate its usefulness, and synthesize information rather than spending the majority of time simply locating appropriate sites on the World Wide Web 
(Hoffman, Kupperman, & Wallace, 1997; Lyons, Hoffman, Krajcik, & Soloway, 1997; Wallace, 1997; Wallace & Kupperman, 1997). 

Fi gure 4. Artemis Collections 
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Students used Artemis as an exclusive interface for accessing the University of Michiganfs Digital Libraiy during the study. This environment provided the scaffolding 
learners required for asking questions, searching for information, and accessing age appropriate collections related to their investigations. Early studies (Lyons, 
Hoffman, Krajcik, & Soloway, 1997; Wallace, 1997; Wallace & Kupperman, 1997) did not utilize Artemis - students used commercially available on-line search 
engines for locating resources. As a result, they were often unsuccessful locating information they could use for their research. It was anticipated that Artemis had the 
potential to influence the degree to which students engaged in information-seeking activities. 

Middle Years Digital Library - On-line Learning Materials 

The Middle Years Digital Libraiy (MYDL) project is a subset of the UMDL project. It focuses on providing printed and on-line learning materials to support student 
inquiry on the World Wide Web. Students utilize these materials as a tool to support their information-seeking activities as they ask question of interest, plan then- 
inquiry, search for information, assess their findings, and create rich representations of their newly constructed understandings. The MYDL project offers a number of 
on-line physical science units for use by teachers and students including astronomy, ecology, geology, and weather. An additional unit, the Scavenger Hunt, is designed 
to help learners who are new to using Artemis, the UMDL, and navigating the World Wide Web. 

Hoffman (1997) describes the iterative nature of design for these on-line learning materials and specific features supporting student inquiry on the World Wide Web, 
These materials include a Home Page (Figure 5) where students begin their investigation, a Road Map that provides an outline view of the major sections of the MYDL 
site, a What to Do page that gives a brief introduction to the science unit and the inquiry process. A References page that allows students to reach the University of 
Michiganfs on-line thesaurus, encyclopedia, and dictionary, a link to Artemis that permits access to the UMDL, and a Share page that allows students to click individual 
icons to reach on-line forms for sharing Driving Questions, sites pertinent to their questions, and comments or questions to other students. 



Fi gure 5. MYDL Ecology Home Page 




The design goal of MYDL on-line learning materials is to provide a common framework for learners as they participate in information-seeking activities across a 
number of content areas, Hoffman (1997a) describes how general design, navigation, content/context, interactivity, and standardization features in the design enhance 
the usability and consistency of these materials. General design issues encompass the overall nature of the on-line environment including ^pearance, apped to users, 
and performance. Through the use of various support structures and location cues, navigation features allow users to browse on-line materials, easing cognitive load 
and disorientation. Content/context includes efforts to create materials which could be part inquiry activities, by framing the problem in an interesting way and 
providing a variety of scaffolds to assist learners as they use on-line resources. Interactive features allow users to engage in the on-line inquiry and utilize learning 
materials by sharing questions, on-line sites, and comments, thereby encouraging collaboration with larger audiences. Finally, standardization refers to the common 
"look and feel" of the materials that distinguishes them from other sites on the Web, 

An example of a powerful feature in the MYDL on-line learning materials is the ability of students to share and inunediately post sites (called cool sites) they find 
valuable to their inquiry. Previous iterations of MYDL materials included text that encouraged students to record interesting sites in a written journal and share them 
with another students in the classroom. This was replaced by an on-line sharing form (Figure 6), which automatically posted reviews of sites to a Web page where 
students could view each others critiques and ideas. 

Fi gure 6. MYDL On-line Sharing Form 
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In addition, more detailed information about the critiqued site is available for student review. Hoffman (1997a) suggested students might benefit from sh^ng their 
ideas and information with others on-line. Although the current system provides only asynchronous sharing (students do not communicate in real-time), it allows 
students to view others' work, provide feedback, and become conifortable with sharing their own ideas. 

Krajcik, Blumenfeld, Marx, & Soloway (1994) describes the benefits of creating a forum where students can try out ideas and challenge the ideas of others. This 
"sharing" can lead to a sense of community where "knowledgeable individuals share information, data, resources and ideas." Student motivation could increase as a 
product of sharing questions, comments, and reviews with a wider audience and viewing work in a public forum such as the Web. 

Students used the Middle Years Digital Libraryis on-line learning materials as an integral component of their infoimation-seekirig activities. This provided an 
environment to gain a visual context for their on-line inquiry, share Driving Questions, sites, comments, and questions, and obtain access to Artemis. Previous MYDL 
studies used earlier versions of these on-line materials, but they have been redesigned to accommodate Artemis and include a number of other improvements. 

Pedagogical Supports 

In addition to the learning environment, this study is also impacted, to a large degree, by the pedagogical supports provided to learners. Previous 
MYDL studies did not provide a substantial amount of guidance for students, although a number of sessions were conducted with teachers chscussing 
the MYDL pedagogical model. In each study, researchers observed that students did not fully engage with on-line inquiry. This study recognises the 
importance of communicating a comprehensive pedagogical model to teachers and students in an effort to promote a high degree of participation and 
thoughtfulness with inquiry-based learning. This section describes how a set of pedagogical principles provided guidance for learners as they engaged 
in inquiry through information-seeking during this study. 

Early studies (Hoffman, 1997a; Lyons, Hoffman, Krajcik & Soloway, 1997; Wallace & Kupperman, 1997) speak to the challenges of implementing 
materials that support students pursuing inquiry-based activities on the Web. Early research efforts by Wallace and Kupperman (1997) show a 
mismatch between the intended goals of the project and behaviors of students as they engage in on-line investigations. From the analysis of data, four 
themes emerged: "1. Students donit explore much. 2. Students tend to seek answers rather than aim for understanding. 3. Students dornt evaluate 
sources: they receive content from on-line sources at face value. 4. Students learn to use Web tools easily, but use them naively." (p. 12). Based on this 
feedback, the MYDL project investigated the dynamics that occurred between teacher and student, student and student, and student and computer. 
They concluded that one factor influencing a studentis actions during on-line sessions was the pedagogical approach employed by the teacher and 
corresponding degree of support received during the inquiry units, both on and off-line. 

In an effort to improve the success of student information-seeking activities, the MYDL project developed a series of print-based scaffolding materials 
to support students (and teachers) during on-line units. Tactics and JSti^tegies Foi^Lpding Q n-Line Investigations (Hoffman & Meston, 1997) is a 
series of booklets providing guidance for teachers who are new to the Web or unfamiliar supporting student-led inquiry. In addition, it contains age 
and reading level appropriate activity sheets for students to use that provide a process model for scaffolding strategies for inquiry (asking, planning, 
searching, assessing, writing, creating). More specifically, these pedagogical support materials included: 

1. Introduction. This section describes the University of Michigam's Digital Library project, the range of people and organizations that can 
participate in an on-line inquiry unit, how the MYDL project utilizes technology, and suggestions for using the Tactics and Strategies manual 
with students. This information assists in providing teachers with a strong foundation of the developmental background of UMDL-MYDL 
projects, and bow they can begin to incorporate student-led inquiry into the classroom. 

2. Investigating on the World Wide Web. This section presents an overview of the conceptual nature of on-line inquiry, a d^cription of the 
Investigation Wheel (Figure 7) - a graphic used to represent various stages of the inquiry process, and suggests ways in which teachers can 
support students. These teacher-based pages provide of general information-seeking strategies (inquiry strategies) students utilize ^ they 
engage in inquiry with on-line resources. In addition, this section introduces specific pedagogical practices to support students during these 
activities. 

3. About the MYDL Science Units. This section lists the main goals of a MYDL science unit, describes how students will take advantajge of 
on-line materials, and suggests a specific unit option depending on the experience of students. These pages are designed to provide initial 
guidance for teachers as they begin to plan and implement Mft)L on-line and print-based materials with students. 

4. Day-by-Day Plans. This section details specific daily plans, associated activities, and recommended assessments for students. It provides 
specific teacher interventions for supporting students at all stages of their inquiry. This information serves as pedagogical support for teachers 
and assists in their instatiation of inquiry-based learning into their classrooms. 

5. Student Activity Sheets. This section contains student oriented activity sheets to support learners as they ask questions, plan inquiry 
activities, search the Digital Library, assess search results, share information on-line, create products, and use MYDL on-line materials. These 
activity sheets provide explicit scaffolding to guide students during all phases of their on-line inquiry. 

6. Appendix. This section provides a glossary of Internet terms, options for student products, an evaluation rubric, student Internet contract, 
and information about the Teaching and Learning Group at the University of Michigan. 

Figure 7. MYDL Investigation Wheel 
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Research on the use of scaffolding to support the leamine process has been well researched and docume n ted (Palinscar & Brown. I£84: Pans, W i ^SQAi P-alinsc^Ti 
1 Q«6: Rosenshine & Meister. 1992: Wnnd. Riimer. & Rnss. 1976V Scaffolding can be viewed as a process of providing decreasing amounts pf g u pp Qft to hglp 
students "bridge the gap between their current abilities and the intended goal of instruction." (Rosenshine & Mei ster. 1992. p. 26). Other d efimhQns include a 
reference to instructional strategies that are both temporary and adjustable that allow learners "t o participate at ever-increasing Jevels of cpmpetgnge." (Palincsar & 
Rrnwn. 1984. n. 1 22V An additional perspective describes scaffolding as ”a process that enabl es a child or novice to solve a problem. catTV QUt a ^k. Of acmevg a SOm 
which would he bevnnd his for her1 unassisted efforts (Graves et al.. 1996. p. 9 0)." Scaffolding matenals appear in many forms ratlfflBg from Pnnt- J ?as^ IXiatcnalS tQ 
modeling behaviors enacted hv the teacher. Rosenshine & Meister (1992) cite that sca f folds can be applied to the teaching of all tvpes of sk ill s and are Oltsn 
indispensable" for teaching high-level cognitive strategies with steps or proc edures that are difficult to specify. 

These scaffolding materials or procedures often begin with simple exercises that allow learners to participate in a (hfficult tasks or activities earlv on in their i nquiry . 
Through a series of closely monitored steps, difficulty is gradually increased as students beco m e more involved with their learning, and finally the support by th^ 
teacher is withdrawn. This scaffolded instruction allows students to gain appropriate experiences and s k ills to increase their cognitive capabibheg tQWPro 

The intention of MYDL Tactics and Strategies materials were to provide this type of scaffolding so stude nts could learn inquiry strategies and bwme siicce3sfuU^ 
asking broad Questions and pursuing their topics using digital resources on the World Wide W eb. For example, students often h^ye difficulty developing g<;m 
"Driving Questions" broad enough and worthy of a sustained investigation over the Period of a few davs. Que s tions like: "Where is the coldest place on gay » 
limit the student to a single response while "Why is Antarctica the coldest place on earth" provides for _ ajnore rich investigation an4 respongg. The acpyity ghe^t^ 
Asking Good Driving Ouesrion<; (Figure RV demonstrates the use of scaffolding as it provides poo r exa inp l es of Driving Que stions, describes thc attnbmc i ^ 
a ppropriate questions, and prompts students to change poor questions into g oo d ones. In addition, this Pnnt-based scaffolding Shggt <igSCnt?gS DQW a Proad 

Driving Question and divide it into smaller segments to ease the task of locating information. Pinallv. th e activity reouircs Stude nt?; aevglop HiTce Possib l e jJnYmg 
Questions for their inquiry. 

Figure 8. Asking Good Driving Questions Activity Sheet 
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However, print-based materials alone do not guarantee students will develop e n hanced inquirv_skills as_teacheis plav a substantial role in thg spaffoldin g proems . 
Rosenshine & Meister (1992^ discuss that scaffolding often includes a substantial am ount of teacher modeli ng, qiscussipn, prompts, gyidanpg. regulation, and 
feedback to be effective. These additional scaffolds provide the suDDort and reinforcement stu dents require as they movg toward COgn i Uve OTwth. 



To help visualize the process students and teachers engage in during a typical M YDL unit, the followuig describes the activities for ^ content uniLthat folio wSJhg 
MYDL Scavenger Hunt. 




1. Before Session 1- Getting Ready. The teacher spends two or three Periods preparing st udents for the inquiry unit t>y 4iscuss»ng gmrgnt tssues tQ bU lld 
interest presenting project goals, and stating assessment issues. Students complete a n activity sheet "ThelDves tl g aftQD Wheel” introducmg the inqmry procgSS 
(ask, plan, search, assess, write, created and develop initial Drivin g Questions with sUDtX)rt from "Asking Goo4 QucSttODS . " 
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lie the activity sheet **U_sing the Road Map” while on-line to become fami liar with the contents of theJ^_YPL 
MYDL site and critique the Questions of their peers. The teacher supports stu dents^ while on-line and leads_a 



Session 2 - Planning the Investigation. The teacher leads a discussion to assist student^ ; in plannin g their inquin^. The ac tivity sheet ^Planning Your 
Tnvestipatinn" prompts students to think through their timeline, and what sources theyjnav require for their Driv ing Quest ion. la addi tion, students create 
Driving Question folders in Artemis and begin to browse the D igital Library for resources. 

4. Session 3 - Browsing with Artemis. Students use Artemis to browse the Digital Library for res ources, collecting initial information for their auesUon, 
StudenK mav refine their Driving Question to better focus their efforts during on-line sessions. Theieacher provides extensive supporLdiinng this phas£_of 
inquiry, and discusses theJtvailabilitv and quality of information availableJbr student questions. 

5. Session 4 - Searching with Artemis. Students become more focused with searches, locating sources that will help provide a solution to their qu^tion. In 
addition, they begin to synthesize information and formulate new understandings. The teacher continues to provide support bv helping stud ent&^tilize Artemis 
to locate appropriate sources of information. 




6. Session 5 - Searching with Artemis. This dayfs activities are similar to Session 4. 

7. Session 6 - Sharing and Critiquing. Students complete the '^Sharing a CoQl^ite” activity sheet and post a favorite siteJhey found usefiiLduring their on-line 
inquiry. Students complete their work on-line bv critiquing sites posted bv their peers^The teacher supports students as they post and critique sites, 

8. After Session 6 - Creating and Evaluating. Students build a product or artifact representing their newlv constructed understandings (i.e. pamphlet, poster, 
newscast^ and mav conduct a short presentation for the class. The teacher evaluates student artifacts utilizin&_ajubric discussed with students at the start of the 
project. 

Although this sequence of activities represent the ideal case, it is understood that teachers responded differently to student needs and made appropriate adjustments. 
However, these print-based support materials provided a starting point where novice teachers could become familiar with MYDL pedagogy and make progress toward 
implementing student-led inquiry in their classrooms. 

This study provides a fine-grained perspective on the emerging science content understandings of learners as a result of their inquiry. The past section described the 
pedagogical supports for this study, initially speaking to the challenges associated with implementing inquiry-based learning using resources from the World Wide 
Web. Using these challenges as a basis, an extensive suite of tools and resources were developed to provide specific support for fostering student-led inqui^. It was 
anticipated these materials would provide the support students required to become competent in participating in various phases of information-seeking activities. 

Methodology 

The challenge of attempting to understand the impact of the use of technological tools for inquiry-based, information-seeking activities in on-line environments 
required a comprehensive research methodology. This methodology expands both the approach and scope of early studies conducted as part of the MYDL project in 
an effort to obtain a more fine-grain perspective of student emerging inquiry strategies and content understandings. To gain insight into the research question, a variety 
of data sources, data collection methods, and analysis techniques were utilized. This section describes the setting, participants, technological tools, tasla, data collection 
methods, and analysis techniques of the study. The outcome of this studyis analysis is to provide both qualitative and quantitative support for describing the range of 
content understandings students developed as a result of their information-seeking activities. These descriptions provide specific conclusions related to depth of 
understanding (recall, explain, articulate, extend) and accuracy of understanding (accurate, partial, none, incorrect). 



Research Setting 

Student participants for this study were enrolled in six grade classrooms at a middle school located in a medium-sized midwestem city, serving approximately 830 
students and employing 40 full-time equivalent teachers. The school draws students from a majority of middle class, and upper-middle class families, with a wide range 
of educational backgrounds. While some parents have advanced graduate degrees (Ph.D,, M.D.), other parents have not completed high school. It is estimated that 35 
percent of parents have blue-collar jobs, and 65 percent have white-collar or professional careers. 20 percent of students qualify for free or reduced breakfast and lunch 
at school. The ethnic diversity of the school is estimated at 80 percent Caucasian, 16 percent African-American, and a 4 percent mixture of other groups. 

The school operates from a modified middle school model where teams of two teachers work with a single group of students for each of their four academic subjects, 
but the curriculum is traditional in terms of its coverage and delivery. Typical students move from class to class for seven periods during the day, but remain together 
as a group for mathematics, science, social studies, social studies, language arts, and content area reading. The remaining period of the day is an elective, where students 
choose computers, foreign language, industrial technology, music exploratory, band, orchestra, choir, or physical education. In total, the average sixth grade student 
sees four or five different teachers each day. 

This niiddle school contains two computer classrooms (lab) located in the media center, with one dedicated for MYDL and other University of Michigan work. 
Although the classroom is rather small, it accommodates 15 student pairs on computers. All computers are Power Macintosh 5260/100 with 13 inch color monitors, 
wired to the Internet via an ISDN line. In addition to the computers, a single low capacity laser printer is available to students. Downloads and uploads between student 
computers and Internet Web sites are normally accomplished in a few second or less, except for peak usage periods during the day when transfer times can be 
extended to 15 seconds or more. Overall, the computer classroom was adequate for the participants and data collection. 

The University of Michigan has a long history of collaboration with this district and this middle school, and the setting was adequate for this study. The school 
provided unlimited access to teachers, students, the computer classroom, and other facilities. As a result, this cooperative nature and willingness to participate allowed a 
rich set of data to be collected. 



Research Participants: Teachers 

Two teachers participated in this study. Nanci. the first teacher, had ten years experience teaching mathematics, science, geography, and foreign language at foe middle 
school level. Her a.ssignment for the length of the study was teaching sixth grade mathematics and science. Nanci has a Bachelors of Arts degree with teaching 
certification in mathematics and science, and a Masters of Arts degree in middle school curriculum, and continues to enrich her education in mathematics an d science 
throu 2 h the participation in summer institutes. Nanci also is involved in a number of extra-curricular activities including coach ing instructio nal swimming, 
synchronized swimming, and life-guarding. In addition, she is a leader-facilitator for the districtfs sixth grade mathematics cuiriculum. a co-ci^culum leader for her 
schoolis mathematics program, and performs consulting outside the district for mathematics curriculum development. Nanci had pyticipated in the Middle Yeas 
Digital Library project for three years, and had used a number of on-line and print-based materials with hetL^dents. During thisJime. she provided substantial 
feedback regarding the design and integration of University of Michigan curricular materials into middle school classrooms. 



The second teacher who participated in this study. Heather. had_three years experience teaching language arts, science^d geography at the middle s chool level, aqd 
three years teaching at a Montessori school. Her assignment during this study wasTeaching sixth grade science. Heather has a Bachelors of Arts degree in psychology 
and English, is certified to teach general science and language arts, and continues to enrich her education in psychology and science through participation in simimer 
institutes. Heather also is involved in a number of extra-curricular activities including Science Olympiad, ski club, and after school tutoring. Heather has participated in 
the Middle Years Digital Library project for two years, and has used a number of on-line and print-based materials with her students. During this time, she provided 
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feedback regarding the design and integration of Artemis, the interface to the University of Michigan Digital Library into middle school classrooms. 

Research Participants: Students 

Eight pairs of sixth grade science students were selected for this study, four pairs from Nanciis class and 4 pairs from Heathen's class. These students were selected 
based on the following criteria: 

1 . Mixed gender. Participants from mixed gender backgrounds provided data to gain insight on the subtleties of male and female engagement with the 
construction of content understandings. 

2. Mixed ethnic background. Similar to gender issues, participants from varied ethnic backgrounds provided a cultural perspective on the issues related to this 
study. 

3. Average to above average ability. Previous studies with MYDL materials utilized participants at low to middle levels of ability, resulting in IMted insight on 
the impact of on-line resources in science classrooms. This study focused on the impact of these materials on ability groupings at a slightly higher level. 

4. Good communication skills. Participants had a tendency to be average or above average communicators, allowing for the maximum possible insight into 
thinking during on-line sessions and interviews. 

5. Willingness to work cooperatively for the entire school year. Since this was a study covering the entire school year, participants agreed to work together for 
the duration of this study. 

Table 1 describes Heathen's and Nancii's students who participated in this study. In addition to the pair number and pseudonym, it contains information on learners! 
gender and ethnicity. 

Table 1 



Student Participants 







Pair 


Student Name 


Gender 


Ethnicity 


Heatherfs Students 


1 


Brett 


M 


European-American 




Cedric 


M 


African-American 


2 


Edward 


M 


European-American 




Kevin 


M 


Asian-American 


3 


Angela 


F 


European-American 




Jamie 


F 


Asian-American 


4 


Brad 


M 


European-American 




Gabe 


M 


European-American 


Nancifs Students 


5 


Britt 


M 


European-American 




Tanya 


M 


African-American 


6 


Kurt 


M 


European- A meri can 




Mike 


M 


European-American 


7 


Grant 


F 


Indian-American 




Robert 


F 


European-American 


8 


Brooke 


M 


African-American 




Karley 


M 


European-American 



Note: Teacher and student names are pseudonyms. 

The number and diversity of participants was an advantage in this study. 'The district and school are familiar and accepting of university projects and recognize the 
mutual benefit they have for students. Teachers had extensive experience facilitating university sponsor^ projects, ^d possessed the expertise to integrate MYDL 
pedagogical practices into their classrooms. Middle school students at this grade level were open to trying new activities and often excited by the prospect of talking to 
other (teachers, parents, researchers) about their learning. In summary, the range of participants provided a rich source of data for this study. 
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Participant Activities 



This study involved students utilizing the MYDL Scavenger Hunt and four MYDL content units (astronomy, ecology, geology, or weather) during the school ye^. The 
Scavenger Hunt, a tutorial-like set of Web pages, is designed as a sequential five day introductory unit to help students become familiar with navigation, exploration, 
and Artemis, the interface to the Digital Library. A series of pages encourages users to interact and become familiar with hypertext, icon, and image map navigational 
tools on the World Wide Web. In addition, students interact with Artemis to create Driving Question folders, perfonn simple searches, and locate specific infonnation 
on a topic of their choice. Subsequent content units provided a series of on-line materials for students to use as they investigated questions related to specific topic 
areas. Pages included a description of the assignment, a link to the Artemis interface, and provisions for sharing and viewing Driving Questions, sites of interest, and 
comments or questions. 

These four content units closely matched the sixth grade science curriculum for the participating middle school and entire district. Modeled after tiie National Science 
Education Standards by the National Research Council ( 1 996), the districtis sixth grade science curriculum consists of four, ten week units covering astronomy, 
ecology, geology, and weather. Each of these units had a defined set of objectives, activities, and assessments teachers and students were expected to complete dimng 
the 10 week period. One requirement involved undertaking a research project, which has been enacted by engaging students in an inquiry-based MYDL unit. This 
replacement for a traditional "book report" has been accepted by the district - the MYDL units were continued for all middle schools for the school year. 

Teachers cooperating in this study utilized the inquiry -based pedagogy outlined in the Tactics and Strategies for Leading On-line Investigations m^ual as a basis 
for implementation with students. Described earlier, this manual provided specific scaffolding for students during each phase of their inquiry by providing activity 
sheets with directions, suggestions, questions, and writing prompts. These scaffolds were built upon as the year progressed with the intention of removing a number of 
them as students become more proficient at various components of inquiry. However, teachers had the freedom to modify and adjust this pedagogy to suit their 
individual teaching styles and the needs of their students. It was not the intention of this study to force a particular pedagogical framework on teachers, but instead to 
allow them to use principles and ideas from the model to assimilate it into their classroom practices. 

Data Sources 

A variety of data sources were used during this study, focusing on the nature of their emerging content understandings. Four types of dam sources were utilized: 1) 
Process Video for gaining insight into student content discussions while on-line and resources they used, 2) Student artifacts for determining how in-process and final 
understandings were represented, 3) Semi-structured interviews for obtaining in-depth perspectives of content understandings, and 4) Classroom-lab video for 
determining the influence of pedagogical practices on students! construction of understanding. 

Process Video 

Information about the studentis activity on-line (e.g. conversations, use of resources) was gathered with a technique called Process Video (Krajcik, Simmons, & 
Lunetta, 1988; Stratford, 1996; Lyons & Hoffman, 1997). Lyons & Hoffman (1997) describe this process where a computer monitor signal is fed to a converter 
module that changes the information to a standard television si^al. This television si^al is sent to a high qu^ity video recorder (Hi8 VCR), where the students! screen 
activity is recorded on tape. In addition, each student wears a microphone, and the audio signal is increased with a mixer/amplifier, and sent to the video recorder where 
it is saved on the audio track of the tape. This technique allows the researcher to both view screen activity and hear conversations of students while they work with 
computer on-line. Process Video was collected for each student pair during all on-line sessions. In total, this study collected 1 84 sessions of process video tapes, 
averaging 35 minutes each. These tapes provided a rich source of data, allowing detailed insight into student conversations and use of on-line ^ources when engaging 
in information-seeking activities. In addition. Process Video captured a fine level of detail that was not possible with field observations or ambient classroom video. 

Artifacts 

A variety of artifacts were collected from students after each unit - completed MYDL activity sheets, on-line postings, final products (i.e. posters, reports, vidwtapes), 
and other writings initiated by the classroom teacher. Activity sheets contained student written responses to inquiry-scaffolding questions (i.e. Bre^ your Driving 
Question down into smaller parts.), specific notes (title, URL, main ideas) about resources encountered, comments about the trustworthiness of a site, and other 
question prompts related to students! inquiry. On-line postings included students! Driving Questions and subquestions, comments and questions to the teacher or 
peers, and critiques of sites utilized during the on-line unit. Artifacts were collected from each student pair for all MYDL units. In total, these includ^ 1 196 pages of 
activity sheets, on-line postings, final projects, and a number of other writings (e.g. journal entries, content summaries, process reflections) These artifacts (along with 
student interviews) provided a basis for the analysis of emerging content understandings of student pairs. 

Semi-structured Interviews 

A number of semi-structured interviews were conducted with student pairs during the study. These short (20 minute) interviews occurred approximately 1 week 
following the end of each on-line unit to assess the accuracy and depth of science content understandings. Interviews consisted of two parts — the first segment 
assessed student understanding of Driving Questions held before the start of the inquiry unit (i.e. What did you know about your question before you started your 
unit? Where did you learn about that?), the second segment probed understandings gained as a result of student inquiry with on-line resources (i.e. Pretend Ifm a 
reporter and want to find out about black holes. What can you tell me about them?). Specific questions were posed to observe if student understanding was limited to 
simple recall, or could be explained, articulated, or extended (What do you mean when you say a star is a middle star? How would that affect us on earfo?) In total, this 
study produced 32 student interviews. This data (along with student artifacts) lends particular support to the content understanding research sub-questions. 

Classroom-lab Video 

The teachens pedagogical influence on students were critical to characterize during this study. Classroom video was recorded for pre-lab, in-lab, ^d post-lab 
discussions and instruction-given by the teacher. These videos occurred in the regular or computer classroom (lab) and captured both the teachens voice and student 
responses. This study collected 99 classroom videos ranging from 30 to 45 minutes each. The classroom video provided a perspective on the type of background, 
support, and instruction the teacher provided to students -- and how they responded to these scaffolds. It is understood that the pedagogic^ influence of the teacher 
may have a substantial effect on student development of content understandings. Given this potential, classroom video has the potential to inform each research 
sub-questions. 

Summary 

The breadth and variety of these sources provided an opportunity for a fine-grain of analysis. Many of the data collection teclmiques have been shown effective in early 
studies - Process Video provides a reliable method in capturing student conversations and activity on-line, and student interviews are critical in assessing emerging 
understandings that may be hidden in artifacts. 

The methodology described in this section are appropriate to answer the questions raised by this study. The techniques presented not only generated a rich collection 
of data, but more importantly furnished an extended view of student performance over the course of the entire school year - a perspective previous studies did not 
provide. The year-long nature of this data will allow a high degree of insight into the research question: What is the nature of learners! science content understandings 
as a result of utilizing on-line resources? 



Data Analysis 



The focus of this study is to provide insight into the depth and accuracy of learners! science content understandings that emerged as a result of utilizing on-line 
resources. This section describes the codng and analysis techniques employed to develop responses to the research question. 
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Validity 

The breadth of data sources utilized during this study permitted an analysis of the research question from a number of perspectives. Analysis performed by 
triangulating these data sources to reduce any biases or limitations of specific methods. It is intended that this triangulation will reinforce the validity of conclusions 
drawn from this study and provide far more insight than previous studies (Hoffman, 1997; Hoffman, Kupperman, & Wallace, 1997; Lyons, Hoffman, Krajcik, & 
Soloway, 1997; Wallace, 1997; Wallace & Kupperman, 1997) that utilized limited data sources. In addition, the use of eight pairs of students and two teachers 
complemented by year-long data collection will enhance the validity of this study -- again, previous research used a limited number of participants for short term 
studies. Other aspects of the studyfs methodology contributed to enhancing validity: Recording and transcribing interviews (instead of writing notes), and recording 
Process Video of students! work on-line (in place of detailed field observations). This combined methodology increased the likelihood of generating a valid description 
of the study, and reduced the possibility of inaccurate or incomplete data. 

Analysis Stage Overview 

Using the discussion of validity as a foundation, this section presents the stages of analysis utilized to explore the specific research question and sub-questions posed 
in this study. Three stages of analysis were conducted on the data collected during this study: 1 ) Data synthesis of Process Video, student interview audio tap^, and 
artifacts to assess content understandings, 2) Evaluation of depth and accuracy of content understandings, 3) Data synthesis and evaluation of classroom-lab video to 
assess possible pedagogical influences. 

Stage 1 : Content Understanding Data Synthesis 

A large amount of data (student interviews, artifacts. Process Video) were collected during the study to provide insight into the depth and accuracy of content 
understandings gained from information-seeking activities. The first stage of analysis involved reducing this data using various coding schemes and analysis 
techniques. It was intended these analyses would provide the bulk of empirical support required to make claims regarding students! emerging content understandings. 

Student Interviews 

Student semi-structured interviews conducted at the conclusion of each content unit provided a fine-grained perspective on the understandings developed as a result of 
information-seeking. Interview data was transcribed and using a perspective of content understanding as described by Perkins, Crismond, Simmons, & Unger (1995). 
Given this conceptualization as a basis, major categories for coding depths of understandings, are illustrated by the following: 

1. Recalling Information. This depth of understanding involves simply stating, repeating, or regurgitating facts from an information source. It is voiced with 
little or no interpretation from the learner, and is often stated in sparse segmented phrases or sentences. For example, '’The moon is a lifeless satellite that 
orbits the earth." 

2. Offering Explanations. This depth of understanding involves telling, describing with examples, pointing out critical features, and responding to new 
situations. It is often voiced in the form of complete Uioughts possessing a high degree of coherence and meaning. For example, "The moon is a lifeless 
satellite because it does not have an atmosphere, and living things need an atmosphere to live." 

3. Articulating Relationships. This depth of understanding involves expressing ideas with rich explanations, relating or linking concepts together, ^d 
illustrating with examples. It is often voiced in the form of multiple sentences that possess a high degree of coherence and meaning. For ex^ple, "The moon 
is similar to other smaller satellites that do not possess adequate gravity to hold in an atmosphere. The larger a satellite is, the more gravity it possesses and the 
more likely it is to contain an atmosphere." 

4. Extending Explanations. This depth of understanding involves expanding concepts to new situations or circumstances, revising information, providing new 
examples, or going far beyond the original source of information. It is often voiced in the form of multiple sentences constructed with a high degree of 
thoughtfulness and meaning. For example, "If the earth had less gravity, there might be a good chance that earth would not contain the type of atmosphere it 
has today. Our planet might be more like the moon without an atmosphere, water, oxygen, or life." 

In addition to these major categories, further coding divided content understandings into distinct areas of knowledge. These areas (interpreted from Bloom, 1974) 
include understandings related to: 

1 . Specific facts and isolated bits of information. For example, "The moonis period of revolution is 27.3 days." 

2. Processes, directions, and movements of phenomena with respect to time. For example, "The moon moves through a series of phases each month - new 
moon, waxing crescent, first quarter, waxing gibbous, full moon, waning gibbous, waning crescent, and back to new moon." 

3. Methods, techniques, procedures utilized to support discovery and inquiry. For example, "The moonfs period of revolution was observed by the ancient 
Chinese who used die information to develop a lunar calendar." 

4. Theories, patterns, principles, and generalizations. For example, "Satellites, like the moon, obey Keplens three laws of planetary motion, and are locked in 
orbit due to their gravitational attraction to a larger object. 

A summary of the coding scheme, shown in Table 2, presents each of the four levels of content understanding, a description of each of the sub-categories for analysis, 
and the associated coding scheme. 



Table 2 

Coding for Depth of Content Understanding 
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Level 


Code 


Description 


Recalling Information 
(stating, repeating) 


RF 


Recall specific facts and isolated bits of information. 




RP 


Recall processes, directions, and movements of phenomena with respect to time. 




RM 


Recall methods, techniques, procedures utilized to support discovery and inquiry. 




RT 


Recall theories, patterns, principles, and generalizations. 


(Xfering Explanations 
(telling, describing) 


EF 


Explain facts, information, and concepts. 




EP 


Explain processes, directions, and movements of phenomena with respect to time. 




EM 


Explain methods, techniques, procedures utilized to support discovery and inquiry. 




ET 


Explain theories, patterns, principles, and generalizations. 


Articulating Relationships 
(expressing, relating) 


AF 


Articulate facts and concepts. 




AP 


Articulate processes, directions, and movements of phenomena with respect to time. 




AM 


Articulate methods, techniques, procedures utilized to support discovery and inquiry. 




AT 


Articulate theories, patterns, principles, and generalizations. 


Extending Explanations 
(expand, revise) 


XF 


Extend facts and concepts. 




XP 


Extend processes, directions, and movements of phenomena with respect to time. 




XM 


Extend methods, techniques, procedures utilized to support discovery and inquiry. 




XT 


Extend theories, patterns, principles, and generalizations. 



Artifacts and Process Video 

Student artifacts constructed at the conclusion of each inquiry unit also provided a fine-grained perspective on the understandings developed as a result of 
information-seeking activities. The physical appearance of these artifacts were described, printed text and presentation dialogue transcribed, and coded using the 
scheme develop^ for student interviews. In addition to final artifacts, in-process artifacts (MYDL activity sheets, on-line postings, journal writings) provided additional 
support for gaining insight on the sense-making process. 

Process Video collected during each content unit provided additional views into how students constructed understandings using on-line resources. Although this data 
contained a wealth of information on how students used technological tools and engaged with information-seeking activities, specific episodes related to content 
synthesis were coded with a scheme similar to student interviews. 

Data Synthesis 

The data sources and coding schemes provided the opportunity to explore each research subquestion in detail to draw out the nature of learners! content 
understandings: 

1. What depth of content understanding (recalling information, offering explanations, articulating relationships, extending explanations) do learners possess 
and communicate as a result of utilizing on-line resources? To what extent are these made visible in learners! products and conversations? These questions 
were explored through the analysis of artifacts (final products), student interviews, and Process Video over the course of the content units. Final products 
allowed a perspective of polished forms of student content understandings and interviews (conducted a week or more after the creation of the final product) 
provided insight into learners! short-term retention of concepts. Also, Process Video allowed a view of how students constructed understandings while on-line. 



2. How accurate are the content understanding (inaccurate, none, partial, accurate) that learners possess and communicate as a result of utilizing on-line 
resources? To what extent are these made visible in learners! products and conversations? These questions were explored through similar analyses of data 
sources as described above. 

Additional analyses were performed to create a representation of student content understanding present after engaging in information-seeking activities. Figure 9 
shows the depth and accuracy of understandings learners! developed and retained as a result of their inquiry. 

Fi gure 9 . Understandings for "How do black holes form?" by Angela-Jamie (pair 3) 
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Stage 2: Content Understanding Evaluation 

The second stage of analysis involved evaluating leameris content understandings for each content unit. The purpose of this stage was gain broad insights into learaersf 
depth and accuracy of content understandings as a result of their year-long information-seeking activities. 

To facilitate this effort, further data analysis was performed on the initial coding and analysis of student understanding shown previously in Figure 9. Divisions for 
depth of understanding (specific facts, processes, methods, theories) were consolidated into a single data point for each category (recall, explain, ^culate, extend) to 
provide a graphical summary (Figure 10). Determination of these summary points were based on the presence of any division of understanding in the category and the 
highest accuracy of understan^ng. However, incorrect understandings were also included in the gr^h along with partial or accurate underst^dings. This represents a 
student possessing an accurate depth of understanding for a single concept in a topic, but at the same time holding possible incorrect conceptions. 

Fi gure 10 . Content Understanding Sununary for Angela-Jamie (pair 3): 
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Further analvsi.s involved developing broad statistics to illustrate the number of units where students achieved various depths of content understanding s (r ecall 
articulate, extendi. In addition, these were separated bv teacher to gain insight into possible pedagogical influences. Table 3 sho wg_an example of the highest depth of 
understanding obtained by students for the four inquiry units completed during the studv^ 

Table 3 



Highest Depth of Understanding (with accuracy) Achieved for Nancifs Students 




http://www.nar8t.org/nar8t/99confdrdncd/hoffmankrajcik/hoffmankrajcik.html 

15 



Thursdsy, July 6, 2000 



Page: 15 





Unit 




Student Pair 


1 


2 


3 


4 


5) Britt - Tanya 


RI 


RI 


RI 


AR 


6) Kurt - Mike 


RI 


RI 


RI 


RI 


7) Grant - Robert 


RI 


RI 


RI 


RI 


8) Brooke - Karley 


RI 


RI 


RI 


OE 



Note: Ri = recalling information, OE = offering explanations, AR = articulating relationships, 

EE = extending explanations. 88% of units resulted in a highest level of understanding at recalling information, 6 % at offering explanations, 6% at articulating relationships, and 0% at extending 
explanations. 

Additional analyses were conducted to determine studentsi accuracy of understandings for the accurate, partial, and incorrect levels. For example. Table 4 shows where 
incorrect understandings occurred in conjunction with, or in place of, partial or accurate understandings. Again, these were separated by teacher to gain insight into 
possible pedagogical influences. 

Table 4 



Units Containing Incorrect Understandings for Nanciis Students 





Unit 


Student Pair 


1 


2 


3 


4 


5) Britt - Tanya 


I 


I 


I 




6) Kurt - Mike 










7) Grant - Robert 




I 


I 




8) Brooke - Karley 


I 




I 





Note: I = incorrect understanding. 44% of units contained incorrect understandings resulting from the use of on-line resources. 

These analyses provided the bulk of empirical evidence to make claims regarding studentsi depth and accuracy of understanding constructed over the length of the 
study. Further insight on possible factors that influenced these constructions were evaluated in the following stage. 

Stage 3: Classroom-Lab Data Synthesis and Evaluation 

Classroom-lab video data collected during the length of the study provided a perspective on how teacher pedagogy may have influenced the degree to which students 
engaged in information-seeking activities. This data was viewed, transcribed, and coded in a number of broad categories including: 

1. Classroom Discussion (General). The teacher leads, moderates, or participates in a discussion about a general classroom topic (not related to the 
investigation). For example: Having a conversation about an mathematics assignment that is due the next day. 

2. Classroom Discussion (General Investigation). The teacher leads, moderates, or participates in a discussion about a general topic related to the inquiry unit. 
For example: Reviewing parts of the Investigation Wheel with students. 

3. Classroom Management. The teacher attempts to manage the behavior and activities of the whole classroom or an individual student. For example, "This 
class is being much to loud... you need to quiet down so we wonit disturb other students in the Media Center," 

4. Question Response. The teacher responding to a specific question from a student. For example, "Yes, that is correct, there are no black holes in our solar 
system." 

5. Review Content. The teacher reviews science content or information learned during a lesson with students. This is accomplished through question-answer 
or discussion with students. For example, "Sue, tell the class what you learned about hurricanes today," 

6. Strategy Instruction. The teacher provides scaffolded instruction to students in an effort to develop or enhance inquiry strategies (related to ask, plan, search, 
assess, synthesize, write, create). This instruction may be visible in the form of presenting or discussing MYDL activity sheets, modeling, reviewing, 
summarizing, making suggestions, etc. For example, "Letfs take a look at this Driving Question... as you can see, it is very dosed-ended. Finding how many 
tornadoes occurred in Michigan is a one word answer. Jamie, tell us how to make it into a good Driving Question." 

7. Student Monitoring. The teacher monitors the progress of students by physically moving to their location and providing assistance, or asking questions to 
the entire class and eliciting a response. For example, "I can see we are having a difficult time connecting to the server today, letfs all click the stop button and 
have just one student try it." 

8. Task Instruction. The teacher provides explicit instructions to students about their immediate task. For example, "I would like all of you to get out your 
journals and find a clean page to write on." 

Table 5 presents a summary of this coding scheme with the event, description, and code for the activities occurring in Heathens and Nancifs classrooms and labs. 
Tables 
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Coding Categories for Classroom-Lab Video 



Event 


Code 


Sub-Code 


Description 


Classroom Discussion (General) 


CDG 




Leading, moderating, or participating in a discussion unrelated to the 
investigation. 


Classroom Discussion (General 
Investigation) 


CDI 




Leading, moderating, or participating in a general discussion related to 
the investigation. 


Classroom Management 


CM 




Attempts to manage the behavior or activities of students. 


Question Response 


QR 




Responding to a question from a student or group. 


Review Content 


RC 




Review science content with students through question-answer or 
discussion. 


Strategy Instruction 


SI 




Scaffolded instruction to develop or enhance students! inquiry strategies. 




SI 


Q 


Related to asking questions. 




SI 


p 


Related to planning. 




SI 


s 


Related to searching. 




SI 


A 


Related to assessing 




SI 


Z 


Related to synthesizing 




SI 


w 


Related to writing. 




SI 


c 


Related to creating. 


Student Monitoring 


SM 




Monitoring and supporting students by providing individual help and 
suggestions. 


Task Instruction 


TI 




Providing explicit instructions to students regarding an immediate task. 



In addition to this coding, general notes (observer comments) were made during the viewing of classroom and lab video to capture how students responded to the 
teacheris scaffolding, directions, and questions. 

The result of this data synthesis was used to evaluate the general pedagogical practices of teachers to determine if they influenced the degree to which learners 
constructed broad and accurate content understandings. This comparison was accomplished through an analysis of coded Classroom-lab video with specific attention 
paid toward the scaffolds teachers provided to assist with synthesizing information found on-line. Summary tables were created to show the similarities or differences 
between teacher supports. 

This section described the methodology utilized for this research and various coding schemes used to organize data to gain insight into the research question and 
subquestions. The range of analysis and breadth of data sources provided an opportunity to gain a substantial amount of insight into the nature of leamersf content 
understandings. This analysis also illustrates the use of data triangulation to gain the greatest possible validity for this study - it is intended this triangulation will 
reduce any biases or limitations from specific methods. In addition, triangulation will reinforce conclusions drawn firom this study and provide a greater depth of 
insight compared to previous MYDL studies that utilized limited data sources. 



Results 



The purpose of this research was to gain insight on the impact of utilizing the World Wide Web as a technological tool to enhance content understandings in 
secondary science classrooms. More specifically, the study focused on a central question: What is the nature of leamersf science content understandings as a result of 
utilizing on-line resources? Two sub-questions relating to this central questions were analyzed using data sources (Process Video, student artifacts, student 
semi-interviews, classroom-lab video) combined with a variety of coding and analysis techniques. The methodology for gaimng insight into the research questions and 
sub-questions involved three distinct stages: 1) Data synthesis of Process Video, student interview audio t^>es, and final project artif^ts to assess cont^t 
understandings, 2) Evaluation of depth and accuracy of content understandings, 3) Data synthesis and evaluation of classroom-lab video to assess possible pedagogical 
influences. 



Data collected from eight sixth-grade student pairs during four Middle Years Digital Library content units provided the basis for m^ng clain^ relative to the nature of 
leamersf content understandings. Of importance were the depth (recalling information, offering explanations, articulating relationships, extending explanations) and 
accuracy of content understandings. The analysis of this data resulted in the following findings: 

1 . The majority of learners constmcted understandings limited to recalling information or offering explanations, and only few demonstrated the ability to 
expand understandings to articulate and extend levels. 

2. The majority of understandings occurred at the partial level. Students more frequently developed limited understandings of concepts and had difficulty 
obtaining accurate conceptualizations. 

3. A number of students developed (or held to) inaccurate understandings, with the majority of these occurring in Nancifs classes. 

4. Clear differences in teachers! pedagogical practices were apparent during the study, with Heather providing more extensive scaffolding. These efforts 
translated into learners who developed more accurate content understandings (no substantial patterns emerged for depth of understandings). 



This section presents the detailed results of this study. Accompanying these results are evidence drawn from artifacts, student interviews, and Process Video to support 
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conclusions. 

Content Understandings: Depth 

Students constructed meaningful understandings as a result of their investigations but were often limited to recalling information or offering simple ej^lanatior^. 
Although student artifacts were represented with factual content, their conversations revealed a much deeper level of understanding. No substantial difference in the 
depth of understanding was noted between participating teachers. 

This study assessed the depth of understandings learners constructed as a result of information-seeking activities with on-line resources, and how they were 
cotnmunicated in final products and conversations. Data from student interviews, artifacts, and Process Video were analyzed and coded into four categories 
(recalling information, offering explanations, articulating relationships, and extending explanations) using techniques described in the methodology section. Table 
6 shows a summary of the highest depth of content understandings Heatherls and Nanciis students developed (with partial accuracy) for the four content units they 
engaged in during the school year. Table 7 displays a similar summary for utiderstatidings developed at the highest level of accuracy. 

Table 6 

Highest Depth of Content Understanding (partial accuracy) 





Unit 


Student Pair 


1 


2 


3 


4 


Heatheris Students 










1) Brett - Cedric 


RI 


RI 


OE 


OE 


2) Edward - Kevin 


RI 


EE 


OE 


AR 


3) Angela - Jamie 


OE 


EE 


OE 


AR 


4) Brad - Gabe 


RI 


EE 


EE 


OE 


Nancits Students 










5) Britt - Tanya 


OE 


EE 


OE 


AR 


6) Kurt - Mike 


OE 


OE 


OE 


AR 


7) Grant - Robert 


OE 


OE 


AR 


AR 


8) Brooke - Karley 


AR 


OE 


OE 


OE 



Note: RI = recalling information, OE = offering explanations, AR = articulating relationships, EE = extending explanations. Partial accuracy was defined as a response that is for the 
most part correct, but contains some element of incorrect information or a hesitancy to respond. 



Table? 

Highest Depth of Content Understanding (highest accuracy) 
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Unit 




Student Pair 


1 


2 


3 


4 


Heatherfs Students 










1 ) Brett - Cedric 






RI 


RI 


2) Edward - Kevin 


RI 


AR 


OE 


AR 


3) Angela - Jamie 


OE 


EE 


OE 


OE 


4) Brad - Gabe 


RI 


EE 




OE 


Nancifs Students 










5) Britt - Tanya 


RI 




RI 


AR 


6) Kurt - Mike 


RI 


RI 






7) Grant - Robert 










8) Brooke - Karley 






RI 


OE 



Note: RI = recalling information, OE = offering explanations, AR = articulating relationships, EE = extending explanations. Accurate was defined as a response absent of incorrect or 
erroneous information. Blank entries indicate students demonstrated partial accuracy only. 

This analysis underwent further refinement and a summary table constructed (Table 8) to show the percentage of Heathens and Nancifs units where students were able 
to obtain accurate and partial levels of understanding during the four content units. Based on this analysis, a number of units had resulting understandings limited to 
recalling information (35%) or offering explanations (32%), and only a few demonstrated the ability to expand conceptualizations to articulate (22%) and extend 
(12%) levels. It is interesting to note that some units (38%) did not produce content understandings beyond partial accuracy. In a comparison of Heathens and Nancifs 
classes, no substantial patterns (for depth of understanding) emerged other than a tendency of Nancifs students to articulate concepts more (16% compared to 6%), and 
Heathens students to extend concepts more (9% compared to 3%). 



Table? 

Summary of Highest Depth of Content Understanding 





Number of Units 






- Heatherfs Students 


Nancifs Students 




Depth 


Accurate 


Partial 

Only 


Accurate 


Partial 

Only 


Description 


Recall 

Information 


4 


2 


5 


0 


Simply stating, repeating, or regurgitating facts from an 
information source. Voiced with little or no interpretation 
from the learner, and is often stated in sparse segmented 
phrases. 


Offer 

Explanations 


5 


0 


1 


4 


Telling, describing with examples, pointing out critical 
features, and responding to new situations. Voiced with 
complete thoughts possessing a high degree of coherence. 


Articulate 

Relationships 


2 


0 


1 


4 


Expressing ideas with rich explanations, relating or linking 
concepts together, and illustrating with examples. Voiced 
with multiple sentences that possess a high degree of 
coherence. 


Extend 

Explanations 


2 


1 


0 


1 


Expanding concepts to new situations or circumstances, 
revising information, providing new examples, or going far 
beyond the original source of information. 



Note: Values indicate the number of units (32 total) where students demonstrated understandings at accurate and partial accuracy. Accurate was defined as a respon^ absent of 
incorrect or erroneous information. Partial was defined as a response that is for the most part correct, but contains some element of incorrect information or a hesitancy to 
respond. 

Given this analysis, the majority of students appeared to construct understandings limited to a basic recall of information or simple explanation. Evidence from student 
interviews illustrate the limits of typical learners. 

Rrptr and Cedric (pair H - Weather Investigation (unit H 
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Interviewer: 


Cedric, what would be another important thing (about sprites)? 






Cedric: 


Like they come in different forms. 






Interviewer: 


How do you mean different forms? 






Cedric: 


Like fireworks or like two people dancing, umm, oh yeah, or red sprites and blue jets. 






Interviewer. 


I need one more thing. 







Brett: 


Probably like their colors. Cause, they have colors white, red, blue and it was pretty cool 
because of them had like blue jets and they would turn into a sprite and a sprite would be 
red and eventually turn into a sprite that was white. 






Interviewer: 


It sounds like you know a lot about what sprites look like. Did you find out about 
anything that causes them? 






Brett: 


It was upper aunosphere electricity, I think, like from lightning. 






Interviewer: 


Do sprites only happen at the top of clouds? Do they ever happen at the bottom of some 
clouds? 






Brett: 


They don’t really happen at clouds, it is from the electricity in the air. It just gives off a 
flash and its umm... they don't really form in the air they just umm... they form in the air 
not near clouds so it's pretty much anywhere. 



This interview demonstrates how students were confined to a simple restatement of facts (with sparse phrases) and could not provide addition^ explanation to sp^ific 
questions. Students could not explain how sprites are associated with thunderstorms clouds and can only be recorded with special earner^ uhliang low hght level 
technology. Also, students assumed sprites occurred anywhere in the sky when research has shown they only occur at the top of clouds. Other interviews provide 
similar insights to the limits of student understanding: 

Edward and Kevin (pair 2^ - Weather Investiparion (unit H 
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Interviewer: 


What did you find out (about hurricanes)? 






Edward: 


We found out that it needs sea water above 80 or 90 degrees and that it is high pressure 
above the low pressure above the water and converging winds. That creates a hurricane and 
they go up like that. 






Interviewer: 


So they go up... 






Edward: 


It becomes like a cycle. They go up and it comes back down and it goes over and over 
again and it turns into a hurricane. 






Interviewer: 


Kevin, anything to add to that? What did you learn about hurricanes? 






Kevin: 


No, not much. Umm... (long pause) 



Although these students have some element of explanation in their responses, they have considerable difficulty moving beyond recalling discr^t information. They 
were not able to explain that hurricanes require three conditions to form (and to strengthen); Warm surface waters, high humidity, and the ability to concentrate heat 
vertically. This simple level of recall was typical of the many learners participating in the study. 

However, some learners developed rich understandings as a result of their efforts on-line and could both articulate and extend their explanations. Evidence from 
student interviews illustrate this depth of understanding: 

An gela and Jamie (pair 3) - Astronomy Investigation (uniL2) 



Interviewer: 


Jamie, tell me what you learned about black holes. 






Jamie; 


I learned that the black hole itself, the reason it is black and you can’t see it is because it 
sucks in light and light cannot escape so you wouldn't see it and x-rays do detect them, but 
you don't see how big, you just see like the gravitational pulls and you can't see the 
particles... and like when they find one they know, they can track down how much 
gravitational pull it has but they don't know how wide it is and they really don't know 
what size. 






Interviewer: 


Why does it pull things into it? 






Jamie; 


I know. The reason it pulls it in is because when the star, like when the super nova occurs 
it blows up and like the dust particles and stuff get like pulled together because of space 
and then after that they just sort of suck everything in because they are use to having a 
gravitational pull in a certain direction so it continues to do it and it sort of makes like a 
funnel kind of shape... and everything sort of is just draining in it. 






Interviewer: 


Anything else you learned about black holes? 






Jamie: 


Well ahh, I wouldn’t want to be sucked in one. It’d be like a painless way of committing 
suicide though because you’d only be alive for like a second. 






Interviewer: 


Why would that be? 






Jamie: 


Well, because it stretches you, it stretches you so much that you’d just be a strand of 
spaghetti and so because it’s stretching you just collapse in on yourself. 
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Interviewer: 


Which would be a bad thing. 






Jamie: 


Yeah, (laughter) 



This interview segment illustrates how some learners could reach beyond merely recalling information and express ideas with relating and linking concepts together, 
expanding concepts to new situations, and going beyond the original source of information. Other examples of this rich content knowledge was gathered through 
student interviews: 

Britt and Tanva (pair 5^ - Geology Investigation (unit 4) 



Interviewer: 


Can you tell me anything more about how to prepare for an earthquake? Youfve all ready 
talked about a lot of things. 






Tanya: 


We found that things that are likely to break, that you shouldn't be near them. This shelf 
has glass and stuff on it, you don't want to stand by it. If you're by your china table or 
whatever, you don't want to be like, "oh, no, not my china" and then be standing by it 
holding it up and then it falls and break on you, everything’s going to break and when you 
try to get pass it your going to cut yourself and get cuts and stuff. 






Interviewer: 


Anything else about preparing for an earthquake? 






Tanya: 


There's a lot of stuff to do. Umm, they said like for, like over there if you have a leaky 
ceiling or if your ceiling are cracking or the walls have little cracks in them, you should 
immediately replaster them and like if there's anything wrong with your floors or like, 
what my parents said is like if you have things that are likely to move, like these tables or 
these chairs or something, for sofas what you should get and like tables, you should get 
those stoppers that you put under them so that they don't slide around. They are kind of 
like rubber so that they don't move around. If you have an earthquake or a tornado or 
something then instead of having all the furniture moving around it just stays. 






Britt: 


Can 1 say something about the tornado? The furniture doesn’t move it just goes phew 
(motions horizontally with hand)! 



An interesting observation made during the analysis of data was the stark difference between the depth of understanding apparent in student artifacts (including 
presentations) and those articulated during student interviews. A review of final products and presentations revealed little about the true depth of leamersi content 
understandings — these artifacts often communicated a simple recall of factual information. Student interviews permitted a fine-grained view of these understandings, 
and concepts were probed to determine true depths of conceptualizations. This was true for all levels of student pairs, including those with the weakest understandings. 
The following example illustrates the contrast between artifact and interview: 

Karley and Brooke (pair 8) have created a simple poster for their weather investigation (What is El Nino?). One section reads: "What is El Nino? El Nino is a disruption of the 
ocean's atmosphere. In other terms it is when the ocean water changes it temperature, sometimes it changes by 1 degree, 2 degrees, and even up to 5 degrees. This change really 
affects the weather sometimes in good ways and other times in bad ways. El Nino can cause severe tornadoes, hurricanes, floods, etc. In good ways it can make your winters 
milder and nicer and springs seem earlier." 

Karlev and Brooke (pair 8) - Weather Inve stigation (unit 2 ) 
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Interviewer: 


I noticed this winter we had a lot of warm days and we really didn't have as much rain or 
snow as we did normally. Why was that, if you said El Nino effected us? What caused us 
not to have as cold or as rainy? 






Karley: 


I think because like the cold weather we usually have was up more towards like Canada 
and the Upper Peninsula and we got what Tennessee and Kentucky usually have for their 
winters and that’s usually just a little snow and warm days and stuff like that. 






Interviewer: 


Oh, I see. So what you’re saying is it just kind of pushed all the weather up. Do you think 
we are going to have to worry about this next year? 






Karley: 


Eh'obably not. Eh'obably the ocean temperature won’t change like it did this year. It will 
just stay at normal temperature. 






Brooke: 


Yeah, it will probably just stay. It will probably go back to its normal temperature. It 
won’t change. 



Although the artifact lists only general facts related to the change in weather patterns, students were able to offer additional explanations during the interview to 
illustrate a deeper understanding of weather phenomena. 

This study assessed the depth of content understandings learners possessed and communicated as a result of their activities on-line. Students demonstrated their ability 
to construct meaningful understandings, but these were often limited to recalling information or offering simple explanations. In addition, interviews proved to be a 
source of leamersi true depth of understanding as artifacts contained only factual information. 

Content Understandings: Accuracy 

Some students developed accurate content understandings from their work with on-line resources. However, a substantial number experienced difficulty 
constructing accurate understandings, and as a result constructed conceptualizations that were only partially accurate, A number of learners developed (or held 
to) inaccurate understandings, A difference was noted in studentsi accuracy levels between participating teachers. 

In addition to obtaining a view into studentsi depth of content utiderstandings, this study assessed the accuracy of understanding learners possessed and 
communicated as a result of using on-line resources. Data from student interviews, artifacts, and Process Video were analyzed and coded into four categories 
(accurate, partial, none, inaccurate) using techniques described in the methodology section. Table 9 shows a summary of the results from this analysis for the four 
inquiry units corresponding to Heatherls and Nancils students 

Table 9 

Summary of Unit Accuracy 





Unit 


Student Pair 


1 


2 


3 


4 


Heatherls Students 










1 ) Brett - Cedric 


P 


pi 


ap 


ap 


2) Edward - Kevin 


a 


ap 


ap 


a 


3) Angela - Jamie 


ap 


a i 


a 


ap 


4) Brad - Gabe 


ap 


a 


P 


ap 


Nancils Students 










5) Britt - Tanya 


ap i 


a p i 


ap i 


a 


6) Kurt - Mike 


ap 


ap 


P 


P 


7) Grant - Robert 


P 


pi 


pi 


P 


8) Brooke - Karley 


pi 


P 


a p i 


a 



Note: a = accurate, p = partial, i = incorrect understandings. Values for content understandings indicate any occurrence of accurate, partial, and incorrect understandings in each content unit and the 
transfer task. 
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The results from this analysis were utilized to create a summary to illustrate the highest accuracy of understanding obtained during the inquiry units. Table 10 shows 
the accurate, partial, and inaccurate levels of understanding for the units completed in both Heatherfs and Nanciis classes. Based on this analysis, it appears most 
understandings occurred at the partial level (44%), followed by the accurate level (39%). This implies students more frequently developed limited underst^dings of ^ 
concepts and had difficulty obtaining accurate conceptualizations. An interesting insight is the presence of a larger percentage of accurate understandings in Heathens 
students (24% compared to 15%) and a higher frequency of inaccurate understandings (13% compared to 4%) in Nanciis students. 

Table 10 

Summary of Content Understanding Accuracy Levels 





Percentage of Coded Understandings 




Accuracy Level 


Heatherfs Students 


Nancifs Students 


Description 


Accurate 


24 


15 


Demonstrated by an accurate response that is absent of 
incorrect or erroneous information. 


Partial 


20 


24 


Demonstrated by a response that is for the most part 
correct, but contains some element of incorrect 
information or a hesitancy to respond. 


Inaccurate 


4 


13 


Demonstrated by an response that is considered 
incorrect or not accepted as factual in scientific terms. 



Note; Values indicate the percentage of understanding accuracy that were coded (54 total) for the 32 units students completed. 

Given this analysis, the majority of understandings learners constructed were at partial levels of accuracy with Heatherfs students demonstrating more accurate 
conceptualizations than Nanciis. In addition, Nanciis students had a high incidence of incorrect understandings. These may have resulted from an incorrect construction 
of understandings, or evidence of previously held conceptions. Data from student interviews lend insight into these understandings: 

Grant and Robert (pair 7) > Astr onomy Investigation (unit 3) 



Interviewer: 


Okay, you mentioned that they (asteroids, comets, meteors) moved at different speeds. 
Why would they move at different speeds? What did you learn about that? 






Robert: 


The amount of energy they have inside of them the nucleus, umm, somedmes if they have 
very much and they’re like really old asteroids or comets then they move really slowly and 
if they’re maybe a newer one, then they go faster. 






Interviewer: 


Where would they get that energy inside? Where did it come from? 






Robert: 


Umm, I think it would come from, I just don’t know like some of the oldest how they 
came to be, you know, comets, so they just knew the energy there but they don’t know 
where it came from or maybe cause they were inside a planet that had energy inside of it 
Maybe it exploded so the pieces of rock had energy inside of it. 



This dialogue indicates students have an incorrect understanding of both the origin and mechanisms of motion for asteroids, comets, and meteors. They failed to learn 
all astronomical objects (planets, asteroids, comets, etc.) that reside in or periodically visit our solar system are in constant motion around our sun due to gravitational 
forces between bodies (i.e. sun and earth, sun and comet). Since there is no friction to disrupt the motion of objects in space, bodies stay in continuous motion. 
Although students spent seven sessions researching this topic (What makes comets, asteroids, and meteors move?), they were unable to construct an accurate (or 
partially accurate) response to their Driving Question. The root of their alternate conception may originate from a connection made from topics learned previously in 
science class. When asked about the source of information on comet motion, Robert replied: 



"Umm, they (the Web) don’t have a lot of information about it but we found out about the energy in science and it didn't exactly say that the energy makes the comet move but 
we just like after planning together what it says and all the things we read that the energy does make it move but it’s also the thing that they're orbiting around.” 

Although Robert had some knowledge of gravitational influences, he admits the "energy inside" has a "bigger effect" on motion. A review of students! final artifact 
(newspaper article) shows similar incorrect conceptions; 

’’Exciting news from NASA today. Two comets that are traveling fast are on track to collide in 5 days at 9:30 pm. If you are outside watching you will be able to see it. 
Sciendsts believe you will see bright sparks of blue. Then there will be a bright flash of red then nothing. The energy in the naiddle of the (sic) is supposed to mix gases 
together and make another comet but nobody in (sic) sure. Comets, like meteors and asteroids are powered by the energy within them." 

Other examples of understandings are evident in an interview conducted with Brooke and Karley (pair 8) after the ecology investigation (What most deadly disease is 
found in water?). When asked if people should stay away from untreated water, Karley replied: 



’’They should try not to drink but if they get a little bit it probably won’t hurt them that much.” 

Also, when asked how people get coliform bacteria, Karley commented 

The coliform bacteria came in rusty pipes and like the other one came from not washing your hands and like eating things that have been like a bug got to and stuff like that. 
Students clearly have incorrect understandings about the origin and dangers of contaminants found in drinking water. These students have not yet developed a correct 
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understanding that bacteria is very small and only microscopic quantities are necessary for the spread of disease. In addition, water contanunation with fecal materi^ 
from sewage systems is the primary cause of the occurrence of coliform bacteria, and not rusty pipes or the presence of insects. The origin of this alternate conception 
could not be traced to a source or associated ideas - however, students mentioned they had no previous understanding on this topic and used only Web-based 
resources to answer their Driving Question. 

This study assessed the level of accuracy learners possessed and communicated as a result of their activities on-line. Although many students developed accurate 
content understandings from their work with on-line resources, a substantial number constructed conceptualizations that were only partially accurate, and some formed 
(or held to) inaccurate understandings. 



Pedagogical Practices: 

A clear difference in pedagogical practices between participating teachers was observed during the study, with Heather providing more extensive scaffolding for 
information-seeking strategies. These differences in practices and scaffolds did not appear to influence leamerls depth of content understandings, but may have 
impacted studentsi accuracy of understanding. 

The extent that participating teachers scaffolded the development of infonnation-seeking strategies (inquiry strategies) with students may have impacted studentls 
accuracy of understanding. As presented previously (Table 9), a larger percentage of accurate understandings occurred with Heatherls students (24%) compared 
to Nancils (15%). Also, Heathen's students had less occurrences of inaccurate understandings (4% compared to 13%). Throughout the school year. Heather had a 
strong focus on the development of good Driving Questions, utilization of tools to support inquiry, and use of on and off-line resources for synthesis. In addition, 
she stressed the ongoing improvement of inquiry strategies and process skills to students, and encouraged them to use their Wonder Books (a journal exclusively 
used for science investigations) as a medium for reflection and synthesis of content. Nanci, in contrast, did not focus as strongly on process skills and did not 
demand the same level of participation from students. Table 11 summarizes the main pedagogical differences in scaffolding information-seeking strategies 
(questioning, planning, tool use, searching, assessing, synthesis, writing, creating) between Heather and Nanci. 

Table 11 

Comparison of Inquiry Strategy Pedagogical Practices of Teachers 



Pedagogical 

Practice 


Heather 


Nanci 


Questioning 

Strategies 


Wonder Books used as the basis for question 
development. Specific lessons targeted attributes of good 
Driving Questions, with substantial rehearsing and 
review. Activities included extensive critique of peersf 
quesdons on-line. Attributes reviewed during the school 
year. 


Quesdons often pulled from recent classroom science 
topics. Presentation of Driving Question characteristics 
with examples and student-teacher discussion. Student 
development of question attributes stressed. Some critique 
of peersf Driving Questions on-line. 



Table 10 - continued 

Comparison of Inquiry Strategy Pedagogical P ractices of Teachers 



Pedagogical 


Heather 


Nanci 


Practice 
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Planning 

Strategies 


Collaborative development of grading rubrics. 
Encouraged students to list planning steps. Focused on 
daily reflection of progress and development of 
sub-goals. Many on-line sessions were highly- directed by 
the teacher. 


Formal siep-by-siep plans required during investigations. 
Little reflection on individual progress at the end of each 
session. Large percentage of on-line sessions are 
specified by goals listed on the board before working in 
the lab. 


Tool Use 
Strategies 


Numerous discussions and explicit instructions on the 
use of Artemis. Public sharing of techniques encouraged 
between groups. Use of Artemis features were somewhat 
monitored and feedback provided. 


Competency with tools assumed from Scavenger Hunt. 
Specific directions given on the use of basic Artemis 
features. Advanced featured provided individually to 
students but not to the group as a whole. 


Searching 

Strategies 


Specific activities provided to help with the development 
of search topics. Stressed locating many different sources 
of information on and off-line. Encouraged students to 
look deep into sites and read text for useful information. 


Search topic ideas constructed as a class and provided to 
students. Individual search topic support provided only 
during on-line sessions. Encouraged students to read text 
for useful information. 


Assessing 

Strategies 


Specific examples and instruction provided to determine 
the usefulness and trustworthiness of sites. Stressed the 
use of viewing the URL (gov, org, com, edu) for 
assessing source quality. 


Site comparison activity (MYDL activity sheet) used to 
illustrate comparing resources for trustworthiness. 
Utilized MYDL Assess activity to scaffold strategies. 
Provided hints on the use of URLs to determine source 
quality. 


Synthesis 

Strategies 


Discussion between partners and groups encouraged, 
making thoughts public important. Stress answer 
construction from many different sources. Lillie direct 
instruction on how to synthesize content using student 
discussion as a basis. 


Conducted activities with concept maps to help with 
studentsf construction of understanding. Minor 
instruction on how to synthesize content through student 
discussion. 


Writing Strategies 


Required students to record detailed navigation steps, 
judge relevancy of information, and provide comment on 
content. Wonder Books utilized extensively as a journal 
for processing thoughts and information. 


Required students to complete MYDL activity sheets 
during on-line sessions and used this task as a method for 
evaluating students. Additional reflection or content 
writing not stressed. 


Creating 

Strategies 


Some freedom given to students on product format. 
Whole class in-depth presentations for each unit included 
content and process. Teacher probed students for 
clarification on content during presentations. 
Question-asking by students encouraged. 


Provided many options for final products, stressed a 
focus on content. Presentations tended to be very short 
with little feedback from the teacher or questions from 
students. 



Although a direct relationship between the scaffolding of inquiry strategies and accuracy of content understandings is ^fficult to establish, Heather did ^mpt to 
engage students in a number of synthesis and writing activities that Nanci did not. These included; digging deep into sites to find pertinent content, utilizing multiple 
sources of information (including print-based materials), paraphrasing information before recording it on paper, using Wonder Books as a method of jouraahng 
thoughts and ideas, being selective about what was recorded on paper, and reflecting on notes (and progress) at the end of most sessions. Perhaps this scaffolding 
required students to process information more thoroughly which enhanced their accuracy of understanding. 

In addition to the scaffolding of inquiry strategies, a comparison of general pedagogical practices provided additional insight into the classroom environment. Table 12 
compares the pedagogical practices (classroom management, task instruction, student monitoring, classroom discussion, question response) of Heather and Nanci. 
This analysis showed how Heather provided more clear expectations for behavior in the classroom, gave more specific instructions and guidance, elicited feedb^k 
from students for the groupfs benefit, and focused on scaffolding the group as a whole. Heathen's classroom could be characterized as more on-task, focused, directed, 
collaborative, and engaging than Nancii's. These differences translated into differences in the engagement with inquiry strategies (and resulting content understandings) 
of Heathen's and Nancii's students during the content units. Based on this analysis, it could be infen-ed the more teachers scaffolded students! development of inquiry 
strategies, the more accurate the resulting content understandings. 



Table 1 1 

Comparison of General Pedagogical Practices of Teachers 
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Pedagogical 

Practice 


Heather 


Nanci 


Classroom 

Management 


Students tend to be quiet and orderly while texher talks, 
most misbehavior is not tolerated and swiftly corrected 
with consequences. Most work sessions are business-like 
and closely monitored. 


Students tend to be quiet and orderly while teacher talks, 
some behavior is tolerated in special needs students. 
Many misbehaviors are verbally corrected. Work sessions 
are often noisy with some off-task behaviors. 


Task 

Instniciion 


Students are given explicit verbal directions in small 
segments. Instructions are very specific and little 
opportunity is given for individual interpretation. 


Students are given broad directions which are often listed 
on the chalk board. Instructions provide the opportunity 
for individual pxing and are open for interpretation. 


Student 

Monitoring 


Students are monitored as a large group with some 
individual support. Feedback and guid^ce are given to 
the group with a high level of interaction between the 
teacher, groups, and peers. 


Students are closely monitored during on-line sessions. 
Feedback and guidance are given individually rather than 
in a large group. 


Classroom 

Discussion 


Although much discussion is from the teacher to 
students, there are many opportunities for inter-group 
discussion, sharing, and critiquing. Pausing for a 
discussion during on-line sessions is common. 


The majority of discussion is from teacher to student, and 
student to teacher. Each student is often called upon to 
reply. Only some discussion is between groups. 


Question Response 


Responses to questions often involve students rethinking 
ideas. Questions are often posed to the entire group for a 
response. 


Answers to questions are given to students directly, with 
some posed to the entire group. 



However both teachers had pedagogical practices that could have been improved upon to foster improved depth and accuracy of content understandings: 1) Although 
teachers expected groups to synthesize content in writing, they did not require them to frequently pause and engage in a healthy discussion about the information -- 
students often wrote notes directly from Web pages with little or no conversation. 2) Teachers did not provide a strong enough critique of leamersi in-process and tinm 
artifacts - students often exerted minimum effort and received good grades for the project. 3) Teachers were not able to effectively sup^rt learuersi sense-making and 
help mediate the formation of understanding ~ some students developed strong incorrect conceptions, or held on to previous understandings when construcUng 
meaning from on-line resources. 

This research focused on the ability of eight pairs of middle school students to engage in information-seeing activities in on-Une environments. Of particular interest 
was the depth and accuracy of content understandings leamersi developed and how the pedagogical practices of teachers influenced their constmcUon, A num r o 
conclusions to questions posed at the beginning of this study were discussed in this section, and a discussion of these results are presented in the following section. 

Discussion 



The purpose of this study was to gain insight into the nature of leamersi content understanding as a result of their use of on-line resources. This smdy incorporat^ a 
methodology that included a number of participants, a variety of data sources collected over an entire schwl year, fine-grained methods for the c^ng ot data, and 
three stages of analysis for obtaining responses to the research question and subquestions. In addition, this study offered extensive technological and pedagogical 
supports to provide a rich learning environment for students. These included: The University of Michigam's Digital Libr^ (UMDL) - a comprehensive set of age and 
content appropriate digital resources, Artemis - a student interface to the UMDL, Middle Years Digital Library (MYDL) On-hne learning Materials - a ^ntextual 
setting with sharing tools to support on-line inquiry, and Tactics and Strategies for Le a ding On-line Investigation s - a set of pnnt-based support materials to assist 
learners with the development of inquiry strategies, and a model for teachers to help instantiate on-line inquiry in the classroom. The learning envirorment was 
carefully constmcted to provide both the framework and resources to support learners in their information-seeking activities on the World Wide Web. 



A number of conclusions were drawn from this year-long study describing how students constructed content understandings from on-line resources. Al^ough a 
number of learners could express understandings with explanations constructed from linked information (articulate), or revise md exppd explanaUons beyond 
original sources of information (extend), the majority of students could only recall basic facts or offer simple explanations. An interesung observaUon from the date 
was the factual representation of understandings apparent in student artifacts - learners rarely went beyond a simple presentation of facts, although their conversaUons 
(student semi-structured interviews) demonstrated a more broad depth of understanding. 

In addition to learners! depth of understanding, perspectives were gained on the accuracy of student learning. Many students demonstrated a high degr^ of accuracy in 
their understandings, but a substantial number experienced difficulty in their sense-making efforts and constructed conceptuahzaUons that were only partially accurate. 
Perhaps the most surprising perspective gained from the analysis was the number of learners who developed or held to inaccurate understandings. 



This study also found clear differences between student pairs, and these variations were attributed to the efforts of the p^cipating teachers. While one teacher 
provided substantial scaffolding to assist students in developing strategies for asking questions, planning inquiry activiti^, using on-hiie tools, searching for^ources, 
assessing sites, writing about information, and creating artifacts, the other teacher provided less support. This difference in support could be attributed to the higher 
occurrence of inaccurate understandings, but the date was not conclusive. 



Given these conclusions, a number of questions, issues, and implications emerge that require careful thought, reflection, and discussion: 



1 . Why did some learners construct accurate and broad understandings while others did not? Was this difference due to student motivation, availability or 
quality of on-line resources, engagement with inquiry strategies, or the degree of scaffolding and support provided by the teacher? 

2. Why did artifacts contain primarily factual content when students were allowed wide latitude in the selection of product formats? Why did learners 
demonstrate broader levels of understandings during short interviews but not when they had multiple days to develop artifacts? 

3. Why did some students develop or hold to inaccurate conceptualizations when immersed in a rich informational environment that provided the potential for 
sense-making from many sources? 

The sections below present a brief discussion of these issues and provide suggestions for educators who are attempting to utilize on-Une environments Uke the World 
Wide Web for student inquiry. 



Fostering In-depth Construction of Understandings 



The majority of students failed to construct the type of accurate and broad understandings one would expect from seven days of on-line inquip^. However, some 
learners did construct and articulate understandings that went well beyond the information contained in a traditional textbook or hbr^ enc^opedia. Insights into tms 
difference may reside in the inquiry strategies students utilized during the school year. A larger study conducted by the author (Hoffman, 1999) suggests students who 
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obtained higher engagement with information-seeking inquiry strategies developed more robust content understandings. In addition, pedagogical practices strongly 
influenced student engagement with inquiry strategies. The notion of pedagogical practices influencing the construction of understanding is support^ by a variety of 
research (Wisnudel-Spitulnik, 1998; Blumenfeld, Soloway, Marx, Krajcik, Guzdial, & Palincsar, 1991; Resnick, Levine, & Teasley, 1991). Cle^ly, if students are to 
take full advantage of on-line resources, they must be appropriately scaffolded and supported to develop strategies for posing worthwhile questions, planning and 
adjusting activities, generating search topics and locating useful resources, utilizing technological tools to extend inquiry efforts, assessing the usefulness and 
credibility of sources and information, synthesizing pertinent facts and concepts, writing and reflecting on information and processes, and creating in-depth products to 
represent their depth of understanding. 

This study did not gather data to assess the impact of learner motivation toward information-seeking tasks, inquiry -based learning, or its impact on the construction of 
understanding. However, motivation may have contributed to the partial accuracy and low depth of understanding demonstrated by many learners during this study. 

For example, although the freedom to select individual Driving Questions was expected to increase the level of learner commitment toward the task, some students may 
not have been motivated to pursue detailed answers to their questions. Past research (Pintrich, Mare, and Boyle, 1993; Kruglanski, 1990; Gamer, 1990) suggests that 
motivation may be a substantial factor associated with conceptual change — students who did not have a strong commitment toward learmng would experience 
difficulty constmcting robust understandings. If educators intend to utilize inquiry -based learning as a vehicle for conceptual change, and have students constmct 
broad understandings, they must (among other things) engage students with the process of inquiry and work hard to sustain high levels of motivation and commitment 
to task. 

The quality and availability of resources could have also contributed to the lack of in-depth understandings. Perhaps students could not locate appropriate resoi^ces to 
support their question-asking or the sites were poorly designed and information was difficult to locate. These factors could have prevented them from constmcting 
responses to their questions. This notion is supported somewhat by a larger study conducted by Hoffman (1999) as he describes how the quality of on-line resources 
(general design, content) did impact some students! understandings during information-seeking activities. However, he suggests that student who made use of 
sophisticated inquiry strategies could benefit from poor quality sites equally as well. These observations suggest a stronger relationship between inquiry strategies and 
content understandings rather than with resource quality. Educators should still create opportunities for students to interact with high quality information by carefully 
prescreening Web sites and creating hypertext links of appropriate resources (in the absence of a digital library) for learners to utilize as starting points for their 
inquiry. 

Developing In-depth Artifacts 

This study found student artifacts revealed little about the tme depth of learners! understanding. In most cases, the conceptualizations presented in posters, 
demonstrations, videos, presentations, and other formats displayed a simple recall of factual content. However, semi-stmctured interviews conducted a w^k following 
the end of a unit provided a more complete view of the content students actually acquired - their explanations went far beyond the depth presented in artifacts. Perhaps 
students focused more on the appearance of products (which was the case for many artifacts) rather than on the representation of content, teachers did not make clear 
their expectations for artifacts, final products were not effectively critiqued for depth of understandings, or students lacked experience in creating rich artifacts 
extending beyond a simple presentation of facts. Although this study did not address these issues, other research provides a persp^tive on the difficulty of desigmng 
assessments to evaluate student understanding. Blumenfeld, Mare, Patrick, Krajcik, & Soloway (1995) describe how teachers (in inquiry-based classrooms) often 
asked students to produce a report or participate in a public performance, but these efforts did not require learners to synthesize information or generate "different 
conceptual representations." Given this perspective, ^ucators need to provide substantial scaffolding and support to improve students! representation of content 
understandings on final products. Classroom teachers need to make clear their expectations for the depth of information required in final products, provide examples 
of exemplary artifacts, give constructive feedback on student work, and engage learners in constructive learning activities to extend beyond the simple recall of 
information. 

However, this study found semi -structured interviews more productive in assessing students! true depth and accuracy of understanding. The interview^ design 
permitted the interviewer to explore detailed aspects of student understanding by rephrasing and expanding questioning to ferret out actual conceptualizations. This 
interview technique is supported by Bogdan & Biklen (1992) as a means of freely probing perspectives and allowing participants to "answer from their own frame of 
reference" rather than one structured by rigid questions. This interview method has critical implications for educators attempting to assess student understanding in 
inquiry-based classrooms. Although extensive interviewing of individuals is difficult with existing time constraints in classrooms, teachers need to find opportunities to 
engage in content specific conversations with students. This will allow teachers to obtain a true perspective on student sense-making that traditional assessments do not 
provide. 

Mediating On-line Learning 

Perhaps the most alarming insight in this study was the number of students who developed or held to inaccurate conceptualizations as a result of utilizing on-line 
resources. Although the information-rich environment of the World Wide Web provided ample access to a variety of resources, it did little to shape and mediate 
student learning. A number of factors may have contributed to these incorrect constructions of understanding: The site may have presented incorrect information, ^e 
information may have been misinterpreted by students, or students may have not extended adequate effort to thoroughly synthesize information. Original perspectives 
by Posner, Strike, Hewson, & Gertzog (1982) speak to the difficulty of sense-making and conceptual change. They describe how learners sometimes possess 
"existing concepts to deal with information" and new ideas are easily assimilated. However, more often the current concepts of students are inadequate (or incorrect) 
and require a "radical form" of conceptual change called accommodation. Posner et al. describe various conditions that must be fulfilled before accommodation is 
likely to occur: 1 ) There must be dissatisfaction with existing conceptions, 2) A new conception must be intelligible, 3) A new conception must appear initially 
plausible, 4) A new concept should suggest the possibility of a fruitful research program (further inquiry). Unfortunately, on-line environments like the World Wide 
Web provide little opportunity for challenging students! existing understandings and promoting opportunities for accommodation. 

This raises a critical (perhaps the most important) issue for the continued use of technological learning tools like the World Wide Web for student content learning. 
Educators might assume a information-rich environment like the World Wide Web would provide ample opportunities for developing in-depth content understandings, 
as students are free to take advantage of the breadth of information it has to offer. However, this is far from the truth. This study observed that although some students 
were thoughtful in their browsing and often paused to comment on or discuss information, more prevalent were students who engaged in "surfing" rather than 
information-seeking — their focus was on moving quickly through resources with the hope of locating specific answers to questions. When information was located, it 
was paraphrased, recorded (often out of context), and used as factual content in their artifacts. 

To develop learners who are information-seekers rather than surfers, educators need to scaffold the search and synthesis strategies of students. Educators should help 
students leam to be more deliberate in the selection of search topics, selective in the choice of sites, thorough in the browsing of information, and thoughtful in the 
synthesis of content. In addition, teachers should require students to occasionally suspend their on-line activities to have a healthy conversation (partner-partner, 
peer-peer, peer-teacher) about the information they are encountering. Through participation in these types of conversations, educators can mediate student learning to 
enhance or correct in-process constructions of understanding. 

This research expanded on early attempts to describe how students interact and leam from on-line learning environments like the World Wide Web. Based on this 
work, it is clear students can benefit from access to on-line resources when extensive support and scaffolding is provided by the teacher, but this is far from automatic. 
Expanded models for technology development, curriculum design, and pedagogical practices are required to successfully instantiate on-line inquiry through 
information-seeking in content area classrooms. Only through careful assessment of these models can the research community provide educators and policy makers 
with concrete guidance for utilizing technological tools like the World Wide Web in today!s schools. 
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